On the Value of Software Testers

by | Syndicated

This was originally published under the title “Considering the Value of Software Testers” in Stickyminds, July, 2014. The original, unedited, version appears below. – Pete

I try hard to learn what other people think about testing and how to do it well.  If you are like me, you have as well.  In doing so, you’ve also heard a variety of answers from gurus, telling us what to focus on. If so, then I suspect you’ll find these ideas familiar:

  • Software testers find bugs;
  • Software testers verify conformance to requirements;
    •
  • Software testers validate functions.
There are different versions of these ideas; they may be expressed in different ways.  Some people focus exclusively on one item.  Some will look at two.  Sometimes these ideas are presented as the best way (or the “right way”) to deal with questions around testing. 
Some organizations embrace one or more of these ideas. They define and direct testing based on their understanding of what these ideas mean.  They insist that testing be done a specific way, mandating practices (or documents) under the belief that controlling practices will ensure maximum effectiveness and best possible results. Less training time and easier switching between projects are two common reasons to do this.
Frankly, even when they work, I find the results unsatisfying. For example, the result of “standardizing” often consists of detailed scripts.  These scripts direct people’s efforts, which often results in actively discouraging questions.  The reasons for detailed scripts are often wrapped around concepts that many in the organization have a very shallow understanding of, such as Six Sigma in software development and repeatability of effort.
In Six Sigma, variation is viewed as the cause of error. A shallow understanding of Six Sigma leads to the understanding that varying from assigned steps in a test document will result in “error” in testing, making variation in test runs a cause of deep concern.
If the “expected results” explicitly state one thing, those executing the tests will soon find themselves looking only that thing. As Matt Heusser has often said (and I’ve stolen the line time and again), “At the end of every expected result is another, undocumented statement that says ‘… and nothing else strange happened’.” 
The point is the obvious solution is to direct people to look at broader aspects than what is documented as “expected results.”   This sets up a conundrum around what is and is not part of what should be looked for.
Many of us would assert that the tester should, out of responsibility and professionalism, track down apparently anomalous behavior and investigate what is going on.  Consider the team reducing variation in a shallow way that has defined steps that take a known period of time to execute. Then add a little time pressure.  What do you think happens when they encounter something that does not fit but is not to be check for explicitly? 
The human mind ignores these types of errors, often the most important errors, or at the very least, a hint that might lead to the most important error.  If you doubt this, then here is an exercise for you: Go to gmail or google and look for the banner ads.  Your mind has been ignoring these for year.  Do you notice how large and prominent then are?  Funny how you don’t notice them unless you look!
Of course management can insist that testers be “professional”and investigate off-script issues, but when the testers follow that advice, they will exceed the allotted time for the “test case.”If part of their performance review and the resultant pay/bonus is tied to those measures, can we really expect them to branch out from the documented steps?
Teams that rely on “click and get reports”automated tools for functional or UI testing are set up for a similar problem. Without careful investigation of the results in both the too and application logs, the software will only report errors in the explicit results. That means the error has to be anticipated in advance in order for the automation code to look for it. 
A Different Way
I’ve explored the consequences of these ideas and have tried them myself in my early career in testing.  I don’t believe they work as broadly as many say they do.  Frankly, they fail my smell test. Can I suggest a fourth definition of testing, perhaps not academically through, but a working definition, based on the things I have seen that actually work?

Software Testing is a systematic evaluation of the behavior of a piece of software, based on some model.

Instead of looking for bugs, what happens if we look at the software’s behavior?  If we have a reasonable understanding of the intent of how the software is to be used, can we develop some models around that?  One way might be to consider possible logical flows people using the software may use to do what they need to do.  Noting what the software does, we can compare that behavior against the expectations of our customers. 

These observations can serve as starting points for conversations with the product owners on their needs.   The conversations can incorporate the documented requirements, of course, along with the product owners’ expectations and expertise.   This means the project team can choose the path they wish to examine next based on its significance and the likelihood of providing information the stakeholders are interesting in evaluating next.  

Instead of a rote checklist, testers working with product owners, development team and other stakeholders can compare their understanding of the software and ask the crucial question of “Will this software meet our needs?” 

Comparing system behavior with the documented requirements means that testers can help initiate and participate in discussions around both the accuracy of the requirements (do they match the expectations) and the way those requirements are communicated, thus helping reduce the chance of misunderstanding.  This helps the Business and Requirements Analysts do a better job writing requirements and position us for conversations around how to make requirements better. 

By changing what we are looking for, from specific items in a check list to looking at overall behavior with specific touch points, we change what we do and how we are considered – moving testing from an activity that has to happen to get the software out the door (a cost center to be minimized) to a value-add activity.  

And You 

If you serve in this industry for any length of time, you have probably felt offended if not insulted as a tester. Perhaps someone who had never done testing a day in their life defined a process for you and you did it while knowing that the work you were doing was low-value and would take too long. Perhaps worse might be to be given a detailed low-variation test plan, measured on time, and also told to investigate – a scenario where you can’t win for losing! 

If that is the case, it might be time to say something like this “Let’s talk about what I do as a tester.” I know, you may be scared, worried about your review. A few testers I know have been fired over this, but that is only a few.  Consider the alternative:  keeping a job you don’t really want to have. 

Sometimes, the way to be most effective at your job is to act as if you don’t care about keeping it.

Piper Kenneth McKay at Waterloo



למה ללכת לכנס?So why attend a conference?

by | Syndicated

לפני קצת יותר משבוע חזרתי מCAST, וחוץ מזה שהיה לי כיף חיים, אני גם חושב שהפקתי מהחוויה הזו תועלת. 
ובכל זאת, כנסים זה יקר, וממילא יש מלא הרצאות באינטרנט, אז למה בכלל לטרוח? אני רוצה לרכז ברשומה הזו כמה נקודות שאולי ישכנעו גם אתכם ללכת. אבל, קודם כל, חשוב לומר שלא כל הכנסים דומים, ולא לכולם אתם רוצים ללכת(“אתם”, במקרה זה, הם הקוראים המיועדים שלי – אלו העוסקים בפיתוח תוכנה. אם יש לכם משהו למכור, יכול בהחלט להיות שתעדיפו ללכת לאחד הכנסים שמבחינתי לא מצדיקים מבט שני). חלוקה אחת בה נתקלתי ונראית לי הגיונית אפשר למצוא כאן (לאלו שמתעצלים לקרוא, או מתעצלים לקרוא באנגלית, החלוקה היא לכנסים מכווני קהילת-מקצוענים, לעומת כאלו שמכוונים לשוק המסחרי וההרצאות בהם הן חצי פרסומת. ההטבות שמקבלים מרצים בכנס יכולות לשמש רמז משמעותי כדי לזהות את מיקום הכנס על הסקאלה). אבל, מתוך הנחה שאתם הולכים לכנס מהסוג הנכון, מה כבר יכול לצאת לכם מזה? 
  • הזדמנות לשמוע על מגוון נושאים מעניינים. כן, נכון. יש מלא נושאים מעניינים באינטרנט, עם וידאו באיכות נפלאה והיכולת לצפות בזה מתי שרק מתחשק לכם ולסנן את הנושאים שמעניינים אתכם בלבד. אבל מתי בפעם האחרונה ראיתם משהו במחשב בלי הסחות דעת באמצע? בלי הפיתוי לשים את השיחה ברקע ולעשות עוד שלושה דברים? זה קורה פחות עם אנשים אמיתיים.  יתרון נוסף שיש להרצאות בכנס הוא שלפעמים אתם נכנסים להרצאה רק כי אתם שם, או כי כולם נכנסים, או כי אתם רוצים לשמוע את הדובר (או הדוברת), ומגלים הרצאה מפתיעה. זה מה שקרה לי  בהרצאת הפתיחה של היום השני בCAST. 
  • הזדמנות לשאו&#15
    00; שאלות     – שמעתם הרצאה ממומחה שמתעסק בתחום שאתם מתקשים בו? לא הבנתם משהו בהרצאה בתחום חדש לכם? יש משהו שנראה לכם חשוב לומר? אם אתם נמצאים בכנס אתם יכולים לקום ולשאול. אם ההרצאה מוקלטת, ההערה שלכם תיכנס לשם לטובת הצופים. אתם גם יכולים לתפוס את הדובר לשיחה קצרה אחר כך. 
  • יש הרבה אנשים מעניינים שלא מרצים בכנס אבל מגיעים אליו. לפעמים תמצאו את עצמכם מדברים במשך שעה או שעתיים עם מישהו שרק פגשתם על נושא שמעניין את שניכם. או שתפגשו מישהו שמומחה בתחום בו אתם מסתבכים קצת (ועכשיו, כשיצרתם קשר, אפשר לשלוח לו דוא”ל עם שאלה קצרה או שתיים, או אפילו לשכור אותו כיועץ אם הוא עושה כאלה דברים). 
  • בכנס יש גם סדנאות – היתרון של סדנאות הוא בכך שאפשר גם לתרגל את הרעיונות החדשים. הסדנאות בדרך כלל לא יוקלטו, וגם אם כן – השתתפות בסדנה זה הרבה יותר מועיל מאשר לצפות בה. לא באמת לומדים הרבה מצפייה בלבד. 
  • לומדים מה קורה במקומות עבודה אחרים. יש לנו (או לפחות לי) מעין תחושה שאנחנו יודעים הכל, ומה שאנחנו רואים מסביבנו זה מה שכולם עושים. אבל המציאות די רחוקה מזה – חלק מהאנשים עושים דברים דומים, חלק נמצאים הרחק מעבר למקום אליו חשבנו שאנחנו רוצים להגיע, ואחרים נמצאים במקום בו היינו לפני חמש שנים. זו הזדמנות ללמוד מה קורה אצלם, ומה הם עשו כדי להגיע לאותו מקום (ואם מדובר בכנס מקומי, אולי גם נרצה לדעת אם מדובר במקום בו תרצו לעבוד). אולי סתם תשמעו במקרה על איזה כלי שמשתמשים בו ומוצא חן בעיניכם
  • זו זריקת מרץ נהדרת – לפעמים העבודה היומיומית יכולה להיות שוחקת, ואם אתם היחידים במקום העבודה שמפגינים עניין בתחום מסויים (האחרים אולי מתעניינים בשקט בלי &#1
    513;אתם שמים לב) זה יכול להיות מעייף. בכנס תמצאו אנשים שאכפת להם מנושא הכנס. להיות בחברת אנשים אחרים שאכפת להם מוסיף המון אנרגיה.
  • זה גם לא מזיק לצניעות – אם אתם אלו שמתעניינים בתחום באופן הכי קולני בסביבתכם, קל מאוד להתבלבל ולחשוב ש”אני הכי טוב בזה”. בכנס יש לכם הזדמנות לראות אנשים טובים לא פחות (חלקם יהיו טובים יותר), וללמוד מהם, או לפחות להפנים שיעור זריז בצניעות.
  • זו הזדמנות לקדם את עצמך – לא כולם מרצים בכנס. למעשה, הרוב אינם מרצים. ועדיין, כנסים מספקים מקום להציג את עצמכם בפני המשתתפים האחרים – בחלק מהכנסים מספקים במה להרצאות ספונטניות מהקהל: אלו יכולות להיות שיחות בזק (lightning talks), או שוק פתוח (שמתקרא בלע”ז open space), או אותה פעילות שאין לי מושג איך לתרגם בשם lean coffee. המון הזדמנויות לדבר. אפשר גם סתם לשאול שאלות מוצלחות בהרצאות, או לגשת למישהו בארוחת צהריים ולהציג את עצמכם.  
  • זה כיף – עדיף לא לשכוח גם את זה. בדרך כלל יהיו אחרי הכנס מפגשים עצמאיים (או מאורגנים) של משתתפי הכנס, או טיול משותף ביום שלאחר מכן, ואפילו במהלך הכנס – אנשים מסביבכם נהנים, וזה מדבק. 
  • יש מה לספר אחר כך בבית – הייתם בכנס? מצויין. אם זה היה כנס טוב, אז שמעתם לפחות רעיון אחד חדש שאפשר לספר עליו בעבודה. זה יכול לשפר את הדרך בה עובדים אצלכם, או לכל הפחות לבנות קצת את המוניטין בעבודה, שגם זה חשוב.
אז איך מוצאים כנס טוב? מתחילים להתערב בקהילה. קוראים קצת תוכן בבלוגים או אפילו בטוויטר, מגיעים למיטאפים (היתרון המשמעותי של מיטאפ הוא בעלות הנמוכה – בדרך כלל המפגש יהיה בחינם, וכל מה שנדרש הוא כמה שעות בערב) ומתחילים להתעניין. מתישהו תשמעו על מישהו שהלך לכנס, או מתכנן ללכת &#1
500;אחד. או מארגן אחד. או שתראו הקלטות של הרצאות משנים קודמות.
————————————————————-

A bit over a week ago I got back from CAST, and had a great time. I also think I benefited from the experience quite a bit.  While a post that will summarize my general thoughts from this conference is still about to come, I want to dedicate this post to list a few reason for why everyone should go to a conference, because, to be honest – going to a conference is not a very obvious choice – it involves work-related stuff outside of work, getting to far places (unless a conference is in your city, you’ll probably have to get to a place which is farther than your office), and even for relatively cheap conferences that happen to be next-door, conferences are still a significant expense. Besides, there’s a ton a free material out in the internet, including talks from conferences. Moreover, unlike a course, conferences don’t usually come with a defined “what do I get out of it” list that can be used to sell the idea to either your manager (who might pay for the event) or even to yourself.
First of all, I want to state the obvious – not all conferences are alike, and you don’t want to attend all of them (Also, by “you” I mean my intended audience – software development practitioners, testers or otherwise. If you sell something, you might prefer the conferences I would tag as “don’t bother”). One way I’ve seen and liked to identify conferences and decide whether you want to go can be seen here. For those of you that want only the short version, The division is between a practitioners-community and almost-a-sales-pitch, and the compensation received by speakers is a good hint).
So, assuming you go to the right type of conference, what do you get out of it?

  • A chance to hear about a variety of interesting subjects– Yes, There are even more interesting subjects out there online, some of them are high-definition videos, and you can watch them at your own time, rewind, skip and see only things that actually interest you.
    But! when was the last time you actually sat to watch something on a computer uninterrupted? or without doing three other things at the same time? With real face-to-face encounters you have less of these distractions. Another advantage in conferences is that you might go to a talk that didn’t consider to be in your field of interest, just because it’s there, or that everyone else are going. This talk might end up opening your eyes to something, or “only” be interesting. This happened to me in the CAST2nd day’s opening keynote. 
  • A chance to ask questions – Attended a talk by an expert in something you’re struggling with? You didn’t understand something said in the talk because you’re new to the field? If you are there you can ask. If the talk is recorded, your question will be there for future viewers. You can also catch the speaker for a longer chat later, maybe over lunch.
  • There are a lot of interesting people that get to conferences and don’t speak in fact, at any given conference, chances are that most of the participants are not speakers. So you might find yourself spending an hour or two talking with someone you just met about a common interest you have, or you might meet an expert in a field you are struggling with (and now that you’ve met, you have a communication channel open and you can ask that expert for help later, or even ask them for some consulting work if they do that sort of thing). 
  • There are workshops – The cool thing about workshops is that you get to actually try out what you are learning. The workshops won’t usually be recorded, and even if they will, participating in one is much more useful than watching it.
  • You get to learn what is happening in other places – It is very tempting (at least for me) to think that we know everything, and that everyone are doing the same things we do, but the reality is quite different. Part of the people are doing similar things, others may be far behind us, or far ahead of us while the rest simply have different conditions and need other things. Hearing that such things exist is quite different than actually meeting someone who is facing a different kind of challenge.  
  •  A great energy boost – chances are that if you’re reading this you care about testing. In that case, you can probably use one hand to count others around you that visibly show the same enthusiasm. Add to this a smidgen of daily routine, and you get that sort of wear and tear that tags along. Attending a conference where almost everyone shares your enthusiasm is very energizing. 
  • A humbling lesson –  If you happen to be the only one who seems to care (see previous bullet), it is quite easy to fall into the fallacy of “I’m the best around here”, which isn’t such a great motivation to keep learning. In conferences you will meet people who are at least as good as you perceive yourself to be (some are better). Seeing them in action, say during a workshop, can remind you that there’s still a lot to aspire towards, and you might get some pointers as well.
  • A chance to promote yourself – Not everyone present at a conference. Yet, conferences are a great place to present yourself and make yourself known to other professionals and build yourself a reputation. Conferences today enable a wide variety of opportunities to make your voice heard – lean coffees, open spaces, lightning talks and so on. Even if there’s no formal place for you to get some spotlight, just chatting with some people makes your voice heard a bit. 
  • It’s fun – Usually there will be a plethora of events after the formal conference. Some of it will be pre-organized, some will be spontaneous, or you might just find yourself in a pub with a small group of people you met at the conference. And even just attending the conference is really fun – people around you will be having a great time, and it’s contagious.

So, how do you find a good conference? You start to get involved in the community (or actually, in a community). You read some content in blogs, maybe even in twitter. Attend a meetup or two (The benefit of meetups, besides being great by themselves, is that the entry cost is very low – most of them will cost you only the time you invest in attending, which will be a couple of hours or so), and you open your ears and eyes. Eventually you’ll hear about someone going to a conference, or wanting to go to one, or sharing an experience of having been to one (in which case you’ll have to wait for the next conference). Another option is to see some videos from previous events.

Tinkerer, Testerer

by | Syndicated

There’s something fun about web development. Something fun with trying to build a website from the ground up or tinkering with a template with it until it fits my needs (like this site), building a Jekyll site from the ground up to display a list of software testing conferences or even a Rails app (I want to build more of […]