Software Automation and the Value Chain: Cost Drivers

The Takeaway: Automation can help lower the costs associated with your software project, resulting in more value and a price advantage in the marketplace

A key component in analyzing the value chain of your project/enterprise is investigating the cost drivers in the value chain. Meaning, your relative cost position (RCP) in the market is the sum total of the cost of performing all the activities in the value chain. The lower the cost of performing activities, if all organizations in the market perform roughly the same activities, the more value can be realized (not to say that decreasing costs always results in more value), and a possible price advantage can result.

Envisioning each activity in producing your software as a cost driver shows us several things:

  1. Where the resources go

    When you start to break down the total effort required to produce your software, you start to see every activity as a step that should add value to the product. Value is “the reflection of the price a firm’s product commands and the units it can sell” (Porter, Competitive Advantage, p.38). The difference between the total cost drivers and the total value of the product is the margin. Examining the cost drivers in your software project reveals what people on the project team spend time on and where the organization spends money.

  2. Over time, which efforts increase and decrease in cost

    Real value can start to be obtained when you look at your project in a longitudinal way. Certainly, there is worth in looking at cost drivers at a specific moment in time, but, as I’ve learned from analyzing software projects, analysis of trends over time can reveal information static analyses cannot. For example, have you implemented a new feature in the past six months, and, since then, the cost of your development has increased? Maybe that’s due to the development team having to go back and correct defects in the system introduced by this new feature. If this feature is adding value over and above its cost, maybe it’s worth keeping. If it doesn’t add value to your application, it may be something you’ll want to review and remove.

  3. Which activities provide the most value

    Each activity in the production of your software comes at a cost, whether it’s  money, time, or both. Using open-source software, for example, may not include an up-front cost, but may involve spending more time in the beginning getting the tools set up. Whatever the costs are, hopefully these tasks all create value in the software. Examining the cost drivers in the software lifecycle can show you exactly where the value is created, and which activities produce the most value.

  4. Which activities can be cut back while still delivering value

    Often, there are tasks where the amount of resources spent can be decreased without negatively affecting the value produced by those tasks. Additionally, there may be tasks that can be cut back or removed completely without affecting the amount of value created in the value chain. This is where efficiency gains are made.

  5. Which activities can be increased to increase value

Sometimes using more resources on certain tasks can result in even more value being produced. Is there a new tool or process you can utilize that, ultimately, creates a more usable or valuable application?

Software automation can be an integral part of value creation for your organization. Automation provides several benefits:

  • It allows an organization to utilize its resources more effectively. For example, automation can be scheduled to run after the close of business each day. Utilizing tools like Selenium and the Selenium Grid, an automation team can set up their grid to run automated tests on the manual quality assurance team’s computers when the QA team is not in the office, getting more use out of otherwise-unused machines, and also freeing up the manual QA team to perform other duties during working hours.
  • Automation teams can reduce the complexity and costs of coordination between connected activities in the value chain. Automation can be set up to, for example, check the status of a development environment for build failures, and notify the right team member if something isn’t set up correctly. No need for the project team to start working on a development environment, only to find out, one at a time, that there’s a defect preventing the team from working on the application, then deciding how to move forward from there.
  • Automation can be timed effectively to produce valuable data. Most software build systems can trigger an automated testing package, so that, when a new build is started, testing can be an automatic part of the build process, testing the new build for failures as soon as possible.

In the next article, we’ll look at cost drivers individually, and how automation impacts each of them in order to produce value and increase the margin of your software application.

UI Automation With C# and Selenium: Adding Packages

The Takeaway: Installing the right packages from the start makes everything else a lost easier.

So far, we’ve created a blank solution with a few projects and added that solution to a repository. Now we need to install the right packages so that when we start coding some tests, everything will be in place and we won’t need to go back and figure out what we’re missing. For our automation, we’ll start out with the following packages:

  • Selenium
  • NUnit
  1. In your automation solution (which you can find the previous post for here), Navigate to Tools > NuGet Package Manager > Manage NuGet Packages for Solution

    Visual Studio - Manage NuGet Packages
    Visual Studio – Manage NuGet Packages
  2. Select the Browse option and search for Selenium 

    Visual Studio - Selenium NuGet Packages
    Visual Studio – Selenium NuGet Packages
  3. For each package Selenium. WebDriver and Selenium.Support, select the package. Notice that the project where our tests live is the only option available installing this package. Remember, since the framework is a Shared Project, there’s no DLL file created for that project, but the files required for the tests will be automatically included when the test project is built 

    Visual Studio - Packages Install in the Test Project
    Visual Studio – Packages Install in the Test Project
  4. Select the Test project (UI Tests in the example above) and select the Install Button 

    Visual Studio - Installing Packages
    Visual Studio – Installing Packages
  5. You’ll probably need to confirm the install of each package as well 

    Visual Studio - Review Package Install
    Visual Studio – Review Package Install
  6. After both the Selenium.WebDriver and Selenium.Support packages are installed, search for and install the NUnit and NUnit .Console packages as well 
    Visual Studio - Installing Nunit Packages
    Visual Studio – Installing NUnit Packages

    Once the NUnit packages are installed, you can select the Installed link and see that several NUnitpackagaes were installed. These packages will be the basis for our framework and tests, but we’ll add more as we progress.

UI Automation With C# and Selenium: Adding Your Project to GitHub

The Takeaway: Using GitHub as your repository for version control makes development of your automation solution MUCH easier!

An important element of your automation project is to make sure you can keep it backed up and available. 

A great way to do that is with GitHub. As a version control system and repository, GutHub allows you to keep track of changes in your automation solution, make the code available to others working on the project, and fork your project in order to further develop the project in new directions.

This article will assume that you:

  • Already understand the basics of version control and repositories. If not, this video and associated YouTube channel can help out there
  • Already have a GitHub username and password. If not, do that here
  • Already have the GitHub Extension installed as part of Visual Studio, If not, here it is. 
  1. Open your Automation solution in Visual Studio. 
  2. Right-click on the solution for the project and select Add Solution to Source Control

    Visual Studio - Add Solution to Source Control
    Visual Studio – Add Solution to Source Control
  3. At this point, Visual Studio will create a git folder in the folder where your solution is saved (which is typically hidden). This folder is needed to keep track of changes to your solution

    Visual Studio - Git Folder Created
    Visual Studio – Git Folder Created
  4. You will also notice that your solution, folders, and files have a blue lock on them

    Visual Studio - Solution Locked
    Visual Studio – Solution Locked
  5. Navigate to the Team Explorer window

    Visual Studio - Team Explorer Home
    Visual Studio – Team Explorer Home
  6. Click the Sync link, and the Publish to GitHub option should display

    Visual Studio - Publish to GitHub
    Visual Studio – Publish to GitHub
  7. Click the Get Started link. Since the repository you just created doesn’t haver a remote version, we can use this tool to create a version on the GitHub website

    Visual Studio - Repository Details
    Visual Studio – Repository Details
  8. Click Publish. Now you should be able to go to your GitHub Profile, click your Repositories link, and view your repo
    GitHub - New Repository
    GitHub – New Repository

    Click here to find the above page, so you can either clone or download the files for this solution as well!

UI Automation With C# and Selenium: Visual Studio Project Types

The Takeaway: Using shared projects with class libraries is a good way to start your Visual Studio automation project.

Setting Up Your Visual Studio Project

One of the initial things you need to decide when you start an automation project is what the structure of your Visual Studio Solution will be. I have worked with several different scenarios, involving various types of projects:

Class Library:

Default project type

Results in the compilation of a DLL (Dynamic Linked Library) that can be ran by Nunit

NUnit Unit Test Project:

Available through the NUnit VS Templates Extension in Visual Studio

Provides code snippets that make getting up and running with NUnit a bit quicker

NUnit VS Templates
NUnit VS Templates

Unit Test Project:

Similar to the NUint VS Templates project template, the Unit Test project template creates a class file with a basic frame of a unit test already built in (including a using statement for the Microsoft.VisualStudio.TestTools.UnitTesting package and properties for a class and method to mark them as a test class and test method). 

I’m sure there are others that are and have been used as well. BUT, one of my favorite project types for testing has become the Shared Project.

Shared Projects:

What I love about Shared Projects is that any code in that project, referenced by other projects, is automatically added to and complies with the other project. So, you can have multiple types of test projects in your solution (one for UI testing, one for API testing, one for database testing, one for security testing), and one single repository for your framework. When you compile the code for that project, the shared code that applies to that project.

How Shared Projects Share Code
Shared Projects

Starting Your Automation Solution

  1. Open up Visual Studio. I will use Visual Studio 2015, but these examples should all work in Visual Studio 2017 just as easily
  2. Navigate to File > New > Project
  3. In the New Project window, select the Templates TreeView > Other Project Types
  4. There should be a Blank Solution template available to you. Select that in the Window, give your solution a name (I’m calling mine Automation), ensure that the Create directory for solution checkbox is checked, and click the OK Button:

    Visual Studio Automation Project - Blank Solution
    Visual Studio Automation Project – Blank Solution
  5. Now, you should have a blank solution with nothing in itNow, we need to add at least two projects to this solution: one Shared Project, and one 
  6. Navigate to File > New > Project
  7. Select the Templates TreeView > Visual C#
  8. Select Shared Project, name it Framework, select Add to solution, make sure you’re in the same location as your blank solution, make sure the Create Directory for solution checkbox is selected, and select the OK Button

    Visual Studio Automation Project - Shared Project

      Visual Studio Automation Project – Shared Project

    Now, you should have a solution with an empty shared project. Time to add the test library:

  9. Navigate to File > New > Project
  10. Select the Templates TreeView > Visual C#
  11. Select Class Library, name it Framework, select Add to solution, make sure you’re in the same location as your blank solution, make sure the Create Directory for solution checkbox is selected, and select the OK Button
Visual Studio Automation Project - Class Library
Visual Studio Automation Project – Class Library

So, you should wind up with a solution with two project templates: a Shared Project for your framework, and a Class Library for your tests. This is the base off of which we will build our automation and framework. Using a Shared Project, we can add whatever we need into the framework portion of our solution, then simply use the necessary components when we build and deploy our tests.

Visual Studio Automation Project - Complete
Visual Studio Automation Project – Complete

A Zip file of this project is available here

Software Automation and the Modern Web: The Web As a Platform

The Takeaway: The modern Web is a platform for user connections, and automation can play an important role in ensuring platform integrity.

The advent of Web 2.0 has altered the way users interact with enterprise-level software and the way enterprises incorporate software into their business.

Tim O’Reilly’s definitional article (2005) describes the shift that started around that time and points to a number of factors that have led to the proliferation of Web applications for business use.

The Web as a Platform

The World Wide Web has become its own platform, upon which developers are developing next-generation Web applications and rich Internet applications. The evolution of the Web as a platform allows developers to think about applications as browser-based services, independent of user operating systems. Some even argue that the Internet browser is becoming the next operating system, capable of delivering applications and extending architectural capabilities in much the same way traditional, desktop operating systems have worked for many years (Wayner, 2013; Garaizar, P., Vadillo, M. A., López-de-Ipiña, D., & Matute, H., 2012).

The success of Google’s Chrome OS points to the
successes companies are seeing with implementing
Internet-as-Platform concepts.

Using the Web as a platform, instead of a specific operating system or specific browser, frees developers from the constraints imposed by utilizing operating systems and browsers, allowing the large amounts of interoperability organizations require from their Web applications. Thinking of the Web as a platform also highlights the network effects possible with Web applications, where the value that the Web application provides the organization using it increases as the number of individuals using the application increases.

What This Means for Automation

Automation plays a valuable role in the Web as a Platform in several ways:

  • The World Wide Web allows us to start abstracting applications from traditional operating systems, but the implementation of the same browser in various operating systems still means there is a need to test those implementations in order to minimize risk. That’s where Sauce Labs, BrowserStack, and the Selenium Grid come into play.

We, as automation engineers, need to understand that, for the web applications we automate, a core strength is an application’s flexibility and “lack of control”.  Once a vendor attempts to control both ends of communication, you lose the primary value of the Web. There are ways to mitigate risk, whether it’s by only supporting particular browsers and not using resources to troubleshoot user issues on other browsers, or by testing your application on multiple browsers/versions/operating systems to make sure your application performs as well as it can on a wide variety of end-user machines. Thankfully, we are also able to mitigate risk with responsive websites that can display information in different ways depending on the browser and the end-user device. Decisions made

Decisions made at the beginning of any software project about browser/device compatibility impact automation effort scope and automation engineers should be in on these conversations from the start.

  • A key tenant of Web 2.0 is that the service improves as the number of users increase. The WWW’s focus is on connections, clients, hosts, and, most importantly, the information that flows between through these connections and between these end-points. Automation’s ability to precisely execute a set of inputs, and then compare the actual outputs with expected outputs helps ensure that all possible execution paths of an information system meet user expectations.

Automation’s Place in the Value Chain

The Takeaway: Automation adds value to your DevOps/SDLC process and can be a large part of your project’s success in the marketplace.

I feel that management theory is not only necessary for understanding a individual’s place on a team, a team’s place in an organization, and an organization’s place in the market. Not only that, but I like reading and talking management theory.

While I do recommend Michael Porter’s Competitive Strategy and Competitive Advantage to anyone wanting to know how to better understand how businesses compete in the market or how their business affects and is affected by internal and external forces, I also understand these books are a bit hard to read through with understanding.

Value Chain

“…strategically relevant activities [disaggregated] in order to understand the behavior of costs and the existing and potential sources of differentiation.”  (Porter, Competitive Advantage, p. 33). 

The way you stay in business is by doing things either cheaper and/or better than its competition. Once you separate and examine each individual activity in your collection of activities performed to design, produce, market, deliver, and support the thing you make, you start to gain an understanding of where the resources go. Where do we spend time that we can cut out? Where do we spending money that makes our product better than our competitors’? This is the (basic) idea behind the aglie manifesto, right? Cut out the things that cost time and money unnecessarily

  • processes and tools
  • comprehensive documentation
  • over contract negotiation
  • following a plan,

and focus on the valuable things

  • individuals and interactions
  • working software
  • customer collaboration
  • responding to change.

Focus resources on the things that matter, and you’re a step ahead of competitors that don’t.

Michael Porter's Value Chain
Michael Porter’s Value Chain

Value Activities + Margin = Value Chain

The total Value Chain is comprised of Value Activities and Margin:

Value Activities

“…physically and technologically distinct activities a firm performs… building blocks by which a firm creates a product valuable to its buyers…” (Porter, Competitive Advantage, p. 38).

There are Primary and Secondary (Support) activities. We’ll focus on support just to keep things simple.

The typical primary value activities do not apply across all industries equally. Manufacturing, for example, relies much more heavily on inbound and outbound logistics than software development, for example. For our purposes, I’ll rely on a classical approach to software value chains provided by Boehm and Papaccio (1987, NTRS), and state that the majority of software development falls under the Operations and Service value activities. 

Operations: Transforming inputs into the final product form (requirements into an application)

Service: Providing service to enhance and maintain the product’s value

Margin

“…the difference between total value and collective cost of performing the value activities…”(Porter, Competitive Advantage, p. 38). 

Margin depends upon the efforts placed in the value chain that add value to a product over and above the costs incurred in the value activities. 

What Automation has to Do With The Value Chain

The good news for automation engineers, is that automation can be a crucial part of the value chain. For any task that can be automated effectively, the cost of that activity decreases significantly. For testing, as an example, instead of using three resources for a week to regression test your application, you can do it overnight, and view the results the next morning. Certainly, there are resources required to develop and maintain the regression testing suite, where the value obtained form those efforts starts small, but will increase over time. However, proper planning and execution can help ensure that value is delivered as quick as possible.

Additionally, automation is helpful in the detection, reporting, and verification of defects. A system defect drives down the “worthwhile-ness” of any product. Good automation efforts discover defects before your customers do, and can, if executed correctly, discover them before manual testing efforts. 

The important thing is to start thinking of any repeated task as something that can be automated. Not every repeatable task should be automated, so discernment is required in order to ensure that the right things are automated. But, progression of automation efforts has been shown to increase the value of the activities you perform in your software development life cycle.

As we continue this series, we’ll delve further into aspects of automation that help deliver value over its costs, how you can implement automation to achieve these goals, and what to look for in order to be successful with software automation.

UI Automation With C# and Selenium, Part I

For the past few years, I’ve struggled with a desire to publish a “how-to” series
on automation. On one hand, there are already LOTS of resources out there, frankly, written by people much better at development and programming than

Selenium
Selenium

I am. Dave Haeffner, for example, does a great job of explaining how to get started in his Selenium Guidebook. And now that it’s available in multiple languages (Java, Ruby, C#, Python, JavaScript), that’s an awesome resource for anyone looking to get started in UI and browser automation.

However, as I’ve grown as an automation engineer, especially in the past year, I still find it hard to find resources that do a great job of explaining how to expand your automation palette, moving beyond the

Nunit
Nunit

basic ideas of UI automation, and including:

  • Data-Driven Automation and Testing
  • Keyword-Driven Automation and Testing
  • API Automation and Testing
  • Security Testing
c-sharp
C#

Furthermore, finding resources that speak to the implementation of these ideas in a C#/.NET environment is even more difficult. 

So, the idea is to show some real-life examples of how to implement an automation solution for testing websites and browser-based applications.

Check out my webiste to get an idea of the tools I use in my own automation.

ASPDOTNET
ASP.NET

Some of the ideas I share are ones that I implement in my own work, some may not be. Some may be ones that I figure out while I’m writing these articles and implement because of them.

If you have ANY questions or want to discuss anything, please feel free to comment or e-mail me.

In Defense of Open-Source

Over the past few months, I have seen some criticism of open-source software testing tools. Joe Colantonio, for example, recently interviewed a guest for his podcast (I cannot remember whom) whose summation of open-source testing tools was (and I’m paraphrasing): Why use an open-source tool when you can purchase a product that does a majority of the set-up work for you?

Additionally, I’ve read several other rejections of open-source tools (like Selenium) for various other reasons: It’s more difficult for unskilled workers to use, there is less support for open-source tools, and there’s always a possibility of closing the source, making it a paid application, or, even worse, stopping the development of the tool altogether.

I will suggest that open-source tools are not only great tools to use but are essential to the continuation of the development of the QA role in software development.

Open-source is not “free”. We know that already. The primary costs, it seems for open-source testing tools are:

 
  • The time it takes users to become familiar with the software
It takes time to learn to use a new tool when a choice is adopted. However, this is also true of paid applications as well. QA and development tools ALL take time to learn how to use correctly. Jira, QTP, QAComplete, even SharePoint all have learning curves, regardless if they are open- or closed-source.
 
  • The time it takes to develop the tool

One primary advantage paid applications say they offer over open-source tools is that so many of the features are already developed, connected, and work out-of-the-box, as it were. This is probably true in most circumstances. For an enterprise to purchase a QA/testing tool, especially a complete testing tool, gets you up-and-running quickly. Open Source tools are not as connected, not as structured, and require more time on the front-end for use.

However, I also suggest this is one of the strengths of open-source. I am a big believer in DIY, and feel that certainly, a user can simply use tools to accomplish work, but a user that understands their tools, understands how they are constructed, knows the inner-workings of their applications, and knows how and why elements are connected like they are, will have more success in the long term. For users that do not understand their tools, any defect in the tool, any diversion from the expected course, or any change in features, is potentially the death knell of the use of the tool. It’s akin to seeing the check engine light on your car and automatically assuming the car is undrivable. There may be a major issue with your car, certainly, but it could just be that the gas cap isn’t secured tight enough. An understanding of your tools allows a user to understand problems when they arise and usually makes for a more complete and confident user as well.

Think about this: If you plan to spend money on a testing tool, would you rather put that money towards the purchase and maintenance of a tool, or toward the salary of a user that can accomplish the required tasks using open-source tools? My money goes to paying trained users that understand the process, understand the tools, and can put the tools to their best use. Invest your money in people, not things.

The argument that open-source tools may be shuttered at any time is also not a very strong argument. The true nature of open-source is collaboration between many individuals. True, using an open-source tool developed by one or two people has the possibility of being abandoned at any time, but the very nature of open-source projects is collaborative. Last time I saw, Selenium Builder has 30 contributors on GitHub (of varying degrees of contribution, of course). Obviously, take care in choosing your open-source tools. Apache isn’t going anywhere anytime soon, right?

Lastly, I would suggest that the future of the QA community lies in the use and development of open-source software. The development of “development”, if you will, includes many attempts at open-source software with contributions from millions of developers, and has resulted in many open-source tools that have become standard technologies: Linux, Apache, MySQL, and PHP, just to name a few. The future of QA is becoming more technical, with the increasing interest in automation and the frameworks that are required in order to successfully conduct an automated testing project. We are only in the beginning stages of this technical revolution, and as open-source QA tools expand, so will the influence and importance of quality assurance in the process of delivering quality software to customers.