In the scramble to get hands on keyboards in a development shop, one of the most important phases of development often gets short shrift. That phase is discovery. I was lucky enough to spend a few weeks with one of our clients working through a formal discovery phase, and was reminded how valuable this time and effort can be. It also got me thinking… How can any developer, regardless of project role, gather the information that normally comes out of a discovery effort?
So what is Discovery, exactly? Discovery is the part of development where you answer a few key and seemingly basic questions about the work you think you want to do. These questions include: Why? What? Who? When? There are a few key outputs of a discovery cycle, including:
- Problem statement – a short, concise explanation of what’s in need of improvement/enhancement/rework
- Solution statement – a short, concise statement about what’s going to be done about the problem
- Scope – a list of things that will be done, as well as a list of things that won’t be done
- Assumptions – any assumptions that have been made about environment, people, technology, complexity, etc. in regards to the project
- Risks – a list of things that can derail the project, which might include company direction, technology changes, dependent projects, or that “simple” external system that you need to integrate but haven’t had a chance to see yet
- High-level time estimates – a high level SWAG (Stupid Wild-Ass Guess) at how long it should take to complete the work, or a time window based on other external factors
- High-level resource needs – a high level list of people (or skill sets) and technology resources (hardware, software) that are needed to complete the project
Notably absent is one thing: How? That’s because the how is design, not discovery. Resist the overwhelming urge to start designing the solution before you fully define the problem. This is the part that I struggle with every time I’m asked to help out with the discovery phase.
A Developer’s Life
Discovery information will come at you in one of two ways: either you’ll be asked to answer the above questions and help define the project, or someone might come to you with all or part of this information and expect you to be involved in the subsequent phases of the project. In both cases, it is absolutely vital that you (Yes, you… the dashing figure reading this post) feel comfortable with the outputs of discovery before you start designing or developing anything. This is known (in my head) as Sneaky Discovery. (Credit to Julie for her invention, the Sneaky Proof of Concept… I’ve adapted the concept for Discovery) It ain’t as evil as it sounds, I promise. Let’s talk about this one for a bit.
I know… it can be uncomfortable to question a project’s intent or setup, especially when you haven’t been asked to. But bear with me for a moment. I’m not asking you to put on your fedora w/optional press pass tucked into the band and annoy the ever loving crap out of the people around you while sporting your best Humphrey Bogart voice. I’m just asking you to take 30-60 minutes and make sure that you feel good about the work you’re about to do. Let’s look at a dramatic reenactment of a common situation:
Scene: John, a diligent data professional is at his desk, working on the latest product release. Enter Manager (Mike).
Manager: Hey! Bob! How are you?
John: It’s, um, it’s John. Not Bob. I’m good. What can I do for you?
Mike: Good, Bob, good. Glad to hear it. I just walked out of a Very Important Meeting with Very Important People, and I have a new set of tasks for you. We’re going to speed up Database X. Also, I have these 17 new reports that need to be developed… um, somewhere. Let me check my notes… Ah, there we are! I’ll e-mail this right over to you so you can get started. How long do you think this will take? Can you have it by next Friday?
John takes a deep breath, and time stops. Enter Mysterious Announcer (MA).
MA: Right now, John is faced with a career conundrum. His fate is in his hands, even though he doesn’t realize it right now. He can either nod and say “Yessir, right on that, sir” and make his manager feel temporarily happy about his decision to hire John, or he can ask a few questions that might make his manager feel a bit uncomfortable in the short term, but will save the company thousands of dollars in the long term. Let’s see how John handles it.
Time restarts. John turns to his Manager.
John: Gee, Mike. That’s a lot to absorb. I have a few questions to ask before I commit to a time estimate. Mind if we sit down and talk through the project together?
Mike: (uncertain) Sure? I think we can do that.
Angels sing, bells ring, beautiful and intelligent women begin to notice John’s unconventional good looks and charming, yet adorably shy, personality
Friends, this is your moment to shine. When faced with a situation like this, it’s up to you to do your own version of Sneaky Discovery and make sure that you’re not diving into the shallow end head-first.
When John gets a chance to talk to his manager, here are a few basic questions he should be asking:
1) Why are we doing this? What is the problem we’re trying to solve here? (Problem Statement)
2) What’s the solution that we’re going with? Is it the only one available? Why did we pick this one? (Solution Statement)
3) What are the things that we’re going to do, and how does each of them help solve the problem we’re fixing? What are we NOT doing? (Scope)
4) Any assumptions about technology direction, complexity, or anything else that led us to the next Friday deadline? (Assumptions)
5) Anything I should know that can cause us to get off track? (Risks)
6) How long did you assume this project would take? Have you made any commitments about delivery? (High-level time)
7) Who else is working on this? Do we have a dedicated development environment? (High-level resources)
8) BONUS QUESTIONS #1: Who is impacted by this change? Is it okay if I check in with them?
9) BONUS QUESTIONS #2: How are we testing this? How are we deploying it?
This conversation will probably take less than an hour for a narrow scope of work. It does two things: 1) you step into the project with more information than you had initially and, 2) you show your manager that you want to understand the work you’re doing. Good managers appreciate this. If you’ve a manager who doesn’t appreciate your questions and desire to understand the work you’re doing, you’ve got a lot to think about, my friend.
Now, the worst case scenario is that John’s manager can’t answer these questions. What to do then? In a perfect world, you pull the comically large brake and hit the giant red STOP! button. Now, it is up to you to decide how to handle the situation. I’ll tell you how I normally handle stuff like this, because it happens… often.
Manager: Um, I don’t really know why we’re doing this. The CTO asked for it, and I’m here to get it done. That’s why I’ve been the vice president of middle management for 15 years, damn it!
Audrey: Okay. I can do this work. However, let me lay out some of the risks for you. We don’t understand why we’re doing it, and we might make poor design decisions because we don’t see the big picture. Also, without a clear understanding of what problem we’re solving, I don’t know how to tell you that we’re done.
Manager: Okay, works for me.
Audrey: Can you put that in an e-mail for me?
If I feel likely to get burned, I put my concerns/feedback in e-mail. If I trust my manager to hear me, I tell him in person. The point is that I never dive into design or development until I’ve either gained broad understanding of the project or at least told someone that I’m working under less than ideal circumstances.
As long as we’re all sharing, let me tell you… we’ve all been there. We’ve all taken the task, done the work, and delivered something without really knowing why. It happens. The trick is to know when you’re dealing with a tough situation and how to mitigate risk to you as well as your team.
You’ll likely find that it is up to you to help flesh out those Assumptions and Risks, and to make suggestions about Scope. As the in the trenches guy, you’re probably aware of issues that your manager isn’t. Don’t be shy about sharing that information. Trust me, ‘tis better to air the dirty laundry at the beginning of the project than to grumble at the end because you “just knew” that this issue was going to come up.
An Aside on Scope
We’ve all heard of it, and if you’ve worked for any amount of time in this business, you’ve lived it. The dreaded scope creep. Many times, scope creep happens because discovery was left wanting. Or, the team isn’t disciplined about sticking to the original scope. The pain of scope creep is never totally avoided, but it can be minimized. Foremost, make sure that you have some scope to work with. Remember how Manager Mike said that he wanted to “speed up Database X”? Wow, that’s a broad statement. He also wants to do it pretty quickly. This means that you probably can’t re-architect the entire system or purchase shiny new hardware to solve this problem. You might end up with something like this in your scope statement:
Task: Speed up Database X
1. Tune top 10 worst performing queries
2. Examine index maintenance plan and add key missing indexes
3. Archive old data that is no longer used and remove from primary database
Out of Scope:
1. Change logical design of the database
2. Upgrade hardware
3. Rework data access layer
On a highly regulated team, this may end up in a formal document. If you’re like many developers and things run a little more fast and loose in your shop, there is nothing wrong with shooting off an e-mail or having a conversation about what can be done in the time you have. You could say, “Mike, we have a week and a half to do this work. Let’s focus on low-hanging fruit and non-invasive changes. When we have more time, we can talk about more aggressive changes”. (Managers love idioms like “low-hanging fruit”) He might say, “Great!”, or he might come back with, “So if we wanted to address your out of scope items, how long would that take?” Both are acceptable responses, and you should be ready for them. Point is, put some boundaries around the work you’re doing, and set expectations about how much can really be done with the time and budget provided. This allows you to be the guy who never says, “No”, but also protects you from unreasonable requirements.
One last thought on discovery within the development cycle. As developers, we want to know everything up front. We want details, and we want them now. Here’s the thing… it ain’t gonna happen. Take a deep breath, accept the unknowable, and focus your Sneaky Discovery efforts on getting some clarity on why you’re doing what you’re doing, what you’re doing (and not doing), and what might throw the project off. The rest will shake out during design and development (and possibly testing), and that’s okay. We’re never going to get it perfectly right, but asking a few of the right questions at the right time can get everyone comfortable with how the rest of the project is going to go.
Now, get out there, get proactive, discover some stuff, and get vocal about how your work fits into the larger picture! I’m rooting for you!