Preliminary work;
Review all data, make sure you have all the required files.
For Crashes- supply node stats, CMEM, CLI getscript, alarm logs and des file. (note that des file and cmem may be in FERs 5.4 and above).
Network diagram/network info will probably be asked for, get it now if available.
Make sure all is up to rev (i.e. software and bootprom rev’s).
Review alarm log, check for config errors or abnormal alarms. If there was CTP connection or abnormal activity before the crash, find out what was being done, it WILL be asked, maybe months later so find out now!
For software bugs, the required files depend on the problem. Should be duplicated with CLEAR instructions.
If the problem has been duplicated, test with most recent release.
Investigate if the problem is already known, search databases (clarify and DDTS), call someone or use the NSD email list.
If you want a quick resolution, provide everything but the kitchen sink. Much time is wasted by going back and asking for more information further down the escalation cycle.
Opening a Change request (Clarify) (customer service database).
From inside vanguard, go to “My Service”, then select Feature Request, Internal Bug & Escalation System”.
From there select “New Escalation”.
Make sure you enter your email address correctly.
Customer Name:
If the customer is from Online or from a distributor, include this and then the actual customer name. (Online/Customer Name or Distributor Name/Customer Name)
Problem description line;
Use actual FER name, like Bus Error, EP: ep_Check() - queue full for EP1
Accurately describe the problem itself.
Write it keeping in mind that this is what
you would like to see for a description in the SRN.
(they do use this for the SRNs if it is understandable)
Think also if you were doing a search years
from now, what keywords you can
use. We can only search on this line
for the problem description.
Check for spelling grammar!
Detailed description;
Copy/paste as much relevant information about
the problem as possible. Do not assume
a follow-up email will make it to the Engineer.
No need to paste the FER or any portions, the
FER will be attached.
Only report one problem per CR/Escalation. If
there are multiple problems or different crashes, each must be escalated
separately.
Only one customer per escalation as well. Too
often the “other customer” may require a different patch base/different
product, or he is forgotten completely.
This also helps with defending the urgency/priority.
Do not use all caps.
1
CRITICAL RED
FLAG SUPER HOT, the president gets paged. If you have questions, call someone
before opening up a sev 1. Hard down,
very frequent crashes etc. A patch is expected for a fix ASAP.
2
MAJOR Problem
severely effects DATA, node crashes. A patch is expected for a fix, sometimes
will be in next service pack.
3
MINOR Typically a stats issue or a MINOR
performance issue. The fix will be in the next major release or future service
pack. (No Patch required)
4
NO CUSTOMER IMPACT,
use for documentation issues only.
(Sev 5 is for feature requests which should be
opened under “Open New Feature Request”).
User impact;
As implies, describe the impact to the
customer this problem causes and frequency if known.
Do not just put “HOT”. Explain why it is
HOT!!!
Use detailed description area if needed.
Do
not put “One time crash” unless you know for sure it is not going to crash
tomorrow or months from now.
Fix required on;
Will the customer upgrade to latest and greatest?,
if so just say latest and greatest.
If the customer will not upgrade, state this
and what base the fix needs to be on and why.
If there are other fixes or pending escalations that will need to be included, enter this info here.
If the problem cannot be duplicated, it must
be indicated so in the detailed description. It is then expected that very
detailed information is available.
ALL CR’s must be worked/investigated by GNE before
being escalated to Engineering.
After finishing this, you should get an email
with the CR number. If you do not, contact Dave Cole, Mike Brogan or Jeff Amaral and we can see if it is in
the “in queue wipbin”.
Now that you have yanked the CR or if it has been assigned to you, keep in mind that you “manage” this escalation throughout its lifecycle as long as you are the owner. It is your responsibility to make sure that it is kept updated and not forgotten.
If there was a clarify case open, when the CR is opened the case
should be closed. The case number should be referenced in the CR. Also, the CR
number should be referenced in the case. The customer should be given the CR
number for reference and be told the case will be closed.
Attaching files
You
can attach the files when opening the
CR on the website, or you can attach
them at any time by going back to the website.
From
inside vanguard, go to “My Service”, then select Feature Request, Internal Bug
& Escalation System”. From there select “Escalation Query”. After you
search and find your escalation click
on the tab “add attachments”. Please be aware that many of the search
fields are case sensitive.
To Escalate to
Engineering
#1, Make sure your name is
selected under the tab “CS owner”.
Instead of “sending to DDTS, we now Dispatch the CR to the Engineering Escalations Queue.
From the DESKTOP pull down, select DISPATCH. Then you should see a list of Queues, select “ Engineering Escalations” and then hit “Dispatch”.
Link depicting Status to Condition States;
http://nabu@misfil6.dma.vanguardms.com/Condition_States.htm
Submitted. (CONDITION STATE IS CR OPEN)
New escalation just opened
(confirmed) and has not been worked yet.
Queued for review.
In queue to be worked.
(see the escalating 101 doc if you
have questions on this one)
You have reviewed the problem, all
required info has been supplied and you are working.
You need to duplicate this in the
lab, but cannot start it at this time.
Trying to Duplicate.
You are actively trying to
duplicate this in the lab.
You have duplicated the problem and
may be testing a work around, testing
newer releases, writing up the duplication instructions etc….
Escalated to Engineering and in the
Engineering Queue to be worked.
Engineer is assigned and working
Engineering has provided a debug or
Eng image for testing or problem analysis.
The customer has installed this code.
Engineering has determined root
cause
Engineering knows how to fix this.
Engineering has posted the patch.
Patch has been sent to the account
team/customer.
Waiting for Customer to
install.
Waiting for Customer to install
the patch
Patch OR CONFIG CHANGE has been
sent to the customer to test.
Customer confirmed OK and
Escalation closed. (CONDITION STATE IS NOW CR CLOSED)
The is closes the CR to customer
service.
Engineering has put this fix in
the next major release.
Engineering has closed this as
being tested and fixed in the major release.
Now if the problem is going to be fixed in the next
major release, normally from “Fix Determined” then next status would be;
Verified, Waiting Next Release
(CONDITION STATE IS CR FIXED)
Engineering has coded the fix and
it is scheduled for the next release.
Engineering has tested that this
is fixed in the next major release
The major release is now shipping.
Customer confirmed OK and
Escalation closed. (CONDITION STATE IS NOW CR CLOSED)
For reference…
Common DDTS States:
New: Just arrived, not assigned, may be waiting for the files to be sent up.
Assigned: It has been assigned to a manger or engineer but not yet being worked
Open: It has been assigned to an engineer and it is being worked.
Postponed: This implies it is waiting for information or an update from the GNE, the account team or the customer. See the last status update or see if a patch has been made available.
Rejected: This means that it has been closed without the need of a new software fix. It could have been a hardware issue, resolved by a previous released patch. It can also be closed this way if needed files or information were never provided.
Resolved: This implies that an Engineer has a fix and will be generating a patch or a Patch has been posted. It is not related to the actual state of the customer. The bug/escalation needs to be put in this state to get a patch generated. It may stay in this state until the next service pack or release is available.
Verified: The fix is in the target release.
Closed: The fix is in and the target release is available.