I don’t think our pages are coming out in syndication, so I wanted to make sure this gets out. Check out our schedule page!
There’s a secret about being a developer that I forget all the time. I know it, and I should remember it, but in the daily drama of life, I tend to forget it. Here it is: Knowing how to do something is the easy part. Knowing what to do… that’s hard.
Technical skills allow me to execute on a plan. The good news is that if I don’t know how to do something, there is a wealth of resources out there to help me out. I can probably pilfer a bit of code from a blog, find a checklist, or even call a friend. Knowing the plan? That’s the hard part.
Brent Ozar (Blog|Twitter) wrote a brilliant post about being a consultant. It was so brilliant and apropos that I e-mailed him to ask for advice on a few things I’m dealing with. He was awesome, and thoughtful, and gave me some great ideas. He even recommended a book, The Secrets of Consulting by Gerald Weinberg. I read it, and I immediately felt better. Everyone should read it. It validated something that had been creeping around in the recesses of my brain: I didn’t have a good plan.
Why not? Well, I’d been so busy executing that I’d forgotten to take a step back and think. It happens to the best of us. I tend to be an intuitive person. I feel like something’s not right long before I can put my finger on it. It makes me crazy. It’s like the intelligent part of my brain is whispering, “Audrey… Audrey… Pay attention. This isn’t working”, while the rest of my brain is totally focused on crossing things off the to-do list. The problem? Maybe the to-do list is dead wrong.
So that’s what development managers and project managers are for, right? They put the plan together. They figure out the what, we figure out the how. Right. Right? Hogwash! Yeah, I said it. Hogwash.
Here’s what I believe. Every person involved in a project, from the college intern to the CTO needs to do a personal assessment of the project they’re on. I’d flat forgotten this personal belief of mine. In the rush to deliver, I’d jumped headfirst into a project without first grounding myself. So, after talking with Brent and reading Weinberg’s book, I assigned myself a task: Assess the Project.
Now, I’m not the first person to do this, and I’m certainly not the first person to talk about this. But I constantly have to remind myself to actually do it. It is a liberating exercise. Putting onto paper what’s worrying me about a project makes it real. If it’s real I can do something about it, or at least see it coming before it smacks me in the face.
I have a few personal rules for my assessments:
1) It is a personal document. For my benefit. I might use it as a reference for later communication, but for now, it’s just me and my stream of consciousness. I don’t worry about sounding negative or hurting feelings. If I think it’s going to be really bad, I might even write it at home and on the personal computer.
2) It is mostly a problem-defining exercise, not a problem-solving exercise.
3) No edits till I’m done. None. No tweaking, rewording, or rethinking. This one is hardest for me. I can’t help myself sometimes, and the urge to soften a harsh word or begin in-line rationalizing is tough to resist.
4) This is not a technical document. It is an emotional document. Everything from “I don’t know how to do X” to “Mr. End User refuses to cooperate” is fair game.
Anyway, here’s my process. I ask myself some questions, and answer them. Really, it’s just a set of lists.
1) What is the current state? – What’s going on in the business that prompted this project in the first place? What needs to be improved/created/maintained? Is the system too slow? Are users complaining? Are we losing customers?
2) What is the desired state? – What does everyone want the world to look like when this project is done? Is capacity higher? Turnaround faster? Errors reduced? Is there a totally new process? Is there a shiny new system?
3) What are the problems? – (Remember, we can use the politically incorrect “problem” because it’s a personal document) What’s keeping us from getting to the desired state? What issues do we keep tripping over? Who’s being difficult or unrealistic? Is the schedule reasonable? What am I awake at 4:00 AM worrying about?
4) What can I fix? – Here, I sort of break one of my rules. I try to identify what I can fix that’s broken. Key point: What, not how.
a. Right Now – What can I do right now without anything else happening first? I don’t worry about time or resources; I just list everything I could theoretically fix.
b. Right After – If I fix the things I could fix right now, what’s next?
c. And Then? – If I can theoretically get through the “Right Now” and “Right After”, what could I do?
TANGENT 1: It’s interesting to see if putting together these three lists naturally gets me to the desired state I defined in List 2. BUT… I resist the urge to force it. Be honest.
5) What is my conclusion? – This is the part where I get to rant. I just start writing about where I think this project is headed. Hopefully, the first 4 lists I’ve put together have helped me get my head on straight. If not, well, that tells me something too. Seriously, I rant. I tell it like I think it is. No one is going to read what I say except for me. Do I believe the project is going to fail? I say it. Then say why. Do I think we need to go in a different direction? I put it down. Do I think I’m failing to deliver? Why? It’s the most liberating part of this process. Feels like confession.
6) What questions do I have? – I read back over my first 5 lists, and start writing down any questions I can’t answer. Doesn’t matter how big or stupid or rude. In the past, I’ve written thing like, “Does anyone care if this project succeeds?”, and “Can we hit deadline if [Name Redacted] keeps screwing up?” I might never ask these questions out loud, but it’s therapeutic to ask myself. I might even come up with a few that need real answers that I can ask in public and look proactive and smart.
TANGENT 2: If I can’t put the 6 lists together off the top of my head, this is a giant, flaming red flag. If I can’t define where we are, or where we want to go, or what I can do to help get us there, I’ve got real problems.
So, I’ve poured my thoughts and worries and soul into answering 6 basic questions. What now? I put it away. I leave it alone to marinate for a day. Then, I open it back up again. I read it and try to see what the basic feel is. Is it optimism or despair? Was I overly negative, or did I apply false optimism to my lists? And, most important, do I see the beginnings of a plan?
Ninety-nine times out of a hundred, I see things I could be doing differently. I can begin to filter out the things I can improve versus the things I have no control over. I usually see a plan emerge. I see a way out of whatever hole I’ve dug for myself (or been thrown into). Most importantly, I have a lodestone in this assessment next time the manager asks me what I think. I’ve already thought about it and put it on paper, and I’m not fumbling around trying to describe some general feeling of “Not Rightness”.
I don’t believe that there are impossible projects, but I do believe that there are impossible plans. More than there should be, actually. I know that Development Managers get sick of hearing us whine about the plan. But, if I can say, “I have a few specific questions and ideas about the plan”, they usually sit up and listen. See, there’s another dirty little secret that my dear friend Josh Lane told me once: We all think there are these experts out there that have all the answers. Guess what? There isn’t. It’s us. We’re it.
Scary as hell? Yes. But it also means that it’s on us as developers to not just solve problems, but to help define them as well. Ask anyone in our business… it really is harder than it looks.
The default behavior of SSIS when an error occurs is for the error to “Propagate” or Bubble up in lay terms to the very top and to stop the package execution. For instance, if I have a dataflow and it attempts to insert a null value into a non-nullable column, the task will error out and the package will stop running.
But of course this behavior of package haltage** is not always desirable. For instance, if you are looping through a directory of xml files and using an xml task inside the loop to validate an xml file against a schema (xsd), a failure (meaning a file failed to validate) can be used as information and as means to direct another desired action. You could move the bad file with a file system task out of the directory and move on to the next file to validate. In this case you wouldn’t want the package to stop.
How do you avoid a package execution haltage? One way is to use the MaximumErrorCount Property available on most tasks and containers. Its default value is 1, meaning at the first error the package encounters, that exceeds the Maximum allowed and therefore it issues a STOP to the package. You can change it to 0 (zero) and that means “error all you want, I will never stop.” The problem with using MaximumErrorCount to try and control package haltage is that errors will bubble up, or propagate to the parent items (sequence containers, for each loops, etc all the way up to the package) and cause the package to stop anyway. The way I have previously dealt with unwanted bubbling was to isolate, as much as possible, the functions for which I would tolerate errors to one package and then set the maximum error count to 0 on all of the items—the child task, the containers it resided inside and the package itself.
Arrows indicate the 3 separate MaximumErrorCount properties which would have to be individually configured to 0 to avoid Package Haltage.
Then John Welch (blog | twitter) *** told me the secret—and now I will share it with you. There is a Legendary, Wily property (it’s actually not a property, but it should be) called Propagate–>On Error , and you can change its setting to False. Once this is done, the errors will not propagate or bubble up and you have a more elegant way to prevent unwanted Package Haltage.
The following is a dialog between Julie1 and Julie2 as we attempted to find this illusive
Here is a play by play:
- Select the task in the control flow for which an error should not halt the package.
2. Go to the Event Handler Tab.
3. Click the Blue text:
4. Watch as the gray screen turns to the familiar pale yellow of BIDS. (you don’t have to drag a single task onto the error handler!)
5. Go to the variables of your package. Click the gray button for system variables. ( I click the column header over the names of the variables at this point to sort them, so that I can finally find this BEAST).
6. Change the default value from True to False.
Ta Da. Now your package won’t halt if an error occurs at this task.
Your photograph of “Propagate” will make you millions from the tabloids and you can quit this data racket and follow your passion to become Lady Gaga’s chief groupie.
**HALTAGE –this word is my invention, unlike Sasquatch, who is TOTALLY REAL AND NOT AN INVENTION.
*** John Welch is completely awesome. Just sayin.
The voting is open this week for the BI Track for SQLRally. SQLRally, a new offering from PASS, is a two day conference which will be held May 11 -13 in Orlando, Florida. I think of it as a cross between the free SQLSaturday and the mega conference PASS Summit in Seattle. This is the inaugural year and the organizers have opened the selection of speakers to the prospective attendees.