This unit will focus on Scrum and how Scrum can be used in your group coursework project.
- Describe what Scrum is.
- Define the main components of Scrum.
- Describe the role of the Product Owner.
- Describe the role of the Scrum Master.
- Describe the main parts of the Scrum Framework.
Scrum is a project management and group working philosophy. For project management, the idea is simple: check your project regularly to see if it is heading in the right direction. It is that simple, but few people manage a project this way. The idea has been around for a while:
- "In preparing for battle I have always found that plans are useless, but planning is indispensable." Dwight D. Eisenhower.
- "Plans are of little importance, but planning is essential." Winston Churchill.
- "All human plans [are] subject to ruthless revision by Nature, or Fate, or whatever one preferred to call the powers behind the Universe." Arthur C. Clarke.
- "Planning is useful. Blindly following plans is stupid." Jeff Sutherland, co-creator of Scrum.
A plan is never right. It will be wrong from the start. So you need to constantly change the plan, so why not build that into your process by monitoring your current plan.
Scrum works in iterations, called Sprints. A Sprint has a fixed length of time (typically a week) where a set of tasks will be completed. The team decide how which tasks they will complete in a week and then analyse how well they did at the end of the week. This allows a team to work towards a goal in increments. The team need to know the goal, see the goal, and work incrementally towards the goal. This is as far as you plan in detail.
And as a plan is always wrong, you will constantly analyse the work to be done. Unforeseen circumstances will arise, so to quote Jeff Sutherland: "Never forget: the map is not the terrain."
Scrum changes how we approach work. It asks the questions:
- What can we change about how we work?
- What is our biggest sticking point?
You should constantly ask these questions when working on a project.
Scrum's core ethos of change is that we no longer look for blame and fault when projects go wrong. It aims to get teams working towards the goal by rewarding the behaviour of collaboration and getting work done.
When selecting tasks to work on, the team prioritises by what returns the most value to the customer. This requires constant communication with the customer. If done well, the team deliver a product that delivers the majority of the value quickly. The Pareto Principle tells us that 80% of the value comes from 20% of the features. If we can deliver that 20% first, we provide the most value to the customer quickly.
Scrum defines an Inspect and Adapt cycle. We are constantly inspecting how we work, and then adapting based on that observation. We want to do work, try something, and check how we do. We want to fail fast so we can fix our problems quickly - spot a mistake and fix it straight away. Otherwise you you will pay for the mistake in the future when you have forgotten the details of the error or work done. Failure is one of the best teachers.
Scrum also deals with waste. Waste is a concept from lean, and we will look at Lean Software Development in Unit 02b. We will talk more about waste then.
Finally, work is changed by focusing on the team. The job of management in the organisation is to free the team to increase the flow of work through the system. Management need to remove obstructions from the team to improve their effectiveness. A standard management approach - adding manpower - has been shown to increase delivery time on software projects, and likely projects in general.
Scrum rewards the team based on work, not on its completion. Goals and results should be celebrated, but they are short-lived. Work, and doing work, should be its own reward. See the process as supporting how you work, making your life easier, and making you more effective. That is the goal of Scrum. The project will be delivered faster because you enjoy the work more. And if you don't enjoy the work, Scrum has mechanisms built in to capture that and fix it.
It is a good point to repeat one of the mantras of the module: education is not a competition. You will get far more out of the educational experience by working together and undertaking the learning. Focusing just on grades will not make you as happy as enjoying the journey to get those good grades and learning from the experience working with others.
In Scrum we use the following terminology:
- Product Backlog.
- Task Board (or Scrum Board, or Kanban Board).
- User Stories.
- Sprints.
- Sprint Planning.
- Story Points.
- Planning Poker.
- Daily Standup.
- Sprint Review and Retrospective.
The Product Backlog is the list of all currently identified tasks. You start by defining all the tasks you can think of at the start of the project. We will look at defining tasks later. The tasks are prioritised by the Product Owner to determine which provide the greatest amount of value quickly to the customer.
To keep track of your tasks we write them on sticky-notes and place them on a whiteboard, something as follows:
We will look at Kanban in Unit 4b to provide more detail. To start with you will define four columns:
- Backlog (or Open Issues).
- Sprint.
- In progress.
- Done (or Closed Issues).
Tasks flow from the the Backlog to Done as they are completed. There are many online tools to achieve this. We will use the one built into GitHub. Trello is another popular tool.
When defining a task it should be part of a User Story. This allows the team to understand why the task is needed. We will look at user stories in Unit 05b. As an example, we might write:
- As a student I want to submit a coursework.
This story has a number of tasks associated with it. We need to upload a file. That file needs to be associated with an assessment. An assessment needs to associated with a module. We need to login to the system. And so on.
A Sprint is a period of time when a team does work. The typical length of time is a week. At the end of the Sprint the product must have moved forward and be usable by the customer. That is, the customer has to see new features added and completed so they can judge them. If the product is not usable at the end of the Sprint you have a problem. If a task is not fully finished at the end of the Sprint then it is not done - the effort was wasted in this Sprint and something else could have been developed and shown to the customer instead.
To plan a Sprint, the team selects tasks they believe they can complete during the Sprint. These tasks are selected from the prioritised task list.
Each task is scored based on an estimated cost. We don't talk about hours, days, or any other concept of time. We are allocating points. And points are a relative score. For example, we could rate our tasks using animal sizes:
- Mouse.
- Dog.
- Deer.
- Hippo.
- Elephant.
- Blue Whale.
Humans cannot judge the time it will take to complete a task. But they can compare sizes really well. For example:
- Student can login (mouse).
- Student can upload a file (dog).
- Uploaded file is associated with an assessment (horse).
- etc.
Each of the sizes is given a value:
- Mouse (1).
- Dog (2).
- Deer (3).
- Hippo (5).
- Elephant (8).
- Blue Whale (13).
This is the Fibonacci sequence, and has been found to be the best approach to estimate relative task size. And we use the entire team to estimate, because collectively you can come to a better estimate. Estimation is done via Planning Poker.
Planning Poker doesn't use animals but numbers - you just need to remember what the numbers represent. We are comparing task size, not determining days of work. Planning Poker card decks are available:
The process is:
- Select a task.
- Everyone picks a card from their deck to score the task, but does not reveal it.
- When everyone is ready, everyone reveals their card at the same time (so no groupthink comes in).
- If the cards are within two of each other (e.g. a 2, 3, and 5 range) then sum the cards and divide by number of voters to get the average. That is your estimated cost.
- If the cards are not within two of each other, then the smallest and largest scorers describe why they gave the points they did. Then a re-vote. Repeat until the group are within two cards of each other, then average.
Remember - this is an estimate, and an estimate for a tiny part of the project. You don't know. But you need to constantly review your scores based on what you have learnt during the project. You will get a better estimate of completion the longer you work on the project.
At the same time every day the team meets for up to 15 minutes. Each person answers the following three questions:
- What did you do since the last stand-up?
- What are you going to do before the next stand-up?
- What is getting in your way?
The daily stand-up builds a routine, and people love routines. It also allows the team to spot blockages. Question 3 is essential. If one of the team has a problem (e.g. "I cannot connect to the SQL database.") then someone might have their solution (e.g. "I know how to do it, let me help."). This is teamwork. Work will be completed faster as tasks will not be blocked.
After a Sprint and before the next Sprint the team undertakes a review and retrospective. How fast are the team working? What went well? What went badly? How can we increase our productivity? In retrospective the team can spot problems and adjust. Remember, inspect and adapt.
The team should ask the following questions:
- Is there anything we can do differently to speed things up?
- Can we offload some Backlog items (i.e. give them to other teams)?
- Can we not do some things (i.e. remove items from the backlog)?
We are also interested in how the team is doing, both collectively and individually. So each person answers the following questions:
- On a scale from 1 to 5, how do you feel about your role in the company?
- On the same scale, how do you feel about the company as a whole?
- Why do you feel that way?
- What one thing would make you happier in the next Sprint?
The team then vote on which improvement to make and execute it in the next Sprint. It needs to be a task with a clear measure of whether it has been successfully done at the next retrospective. Doing this, and doing it consistently Sprint-to-Sprint will mean the team learns, evolves and improves as it works.
A team must have the necessary skills to get the work done. A team should contain all the members needed to deliver the project/product, but should be seven people (plus or minus two) in size. Any larger will create too many communication channels. Smaller teams (as you will have in the module) is fine for smaller projects.
By all skills, we mean everything. Remember the DevOps intersection:
By Devops.png: Rajiv.Pant
derivative work: Wylve - This file was derived from: Devops.png: , CC BY 3.0, Link
IT Operations are part of your team. They help you deliver your product. Your team might include sales, admin, or other people. The team needs all the skills for your product/project.
Also, a team member should only be a member of that one team. No other. If a team member is dividing their attention across multiple projects/products then they aren't committed to either.
Key to successful Scrum is being a good member of the team. It is working with others and seeing that as the goal, not your own outcomes. The team also needs to be happy. And plenty of research has been done on what it takes to make someone happy:
- Autonomy to make decisions on what to do and how to do it.
- Mastery of the what they are doing.
- Purpose, or a reason why they are doing something.
There are areas where actions taken are for individual and short-term benefit. This is not a sustainable model as it does not consider what is beneficial for everyone. It is harmful - individuals crashed the global economy in 2008 for personal, individual, short-term gain. Having no thought for others is not good.
Carlo M. Cipolla, an economic historian, defined the Laws of Stupidity. Cipolla defined the following taxonomy:
Don't be a bandit. If you are only working for yourself you are not a team member. Remember, education is not a competition. Work together and you will learn more and get better results in the long-term. Industry wants team players, not individuals.
Also, don't be a passive team member. Passive means not engaging with the team and the work being undertaken. It is not only lazy, but also hurts the rest of the team as they get annoyed and become less productive. As a team, if passivity is spotted it needs to be stopped immediately.
However, do not get caught in blaming others for perceived character traits. Doing so is the Fundamental Attribution Error and we all suffer from it. If you make a mistake, you can provide external reasons for why it happened. When others make mistakes, you will likely blame some characteristic of the person. This is the Fundamental Attribution Error in action. Be aware of it and be supportive of others in the team.
However, do not let a team member cause emotional waste generated by a team member acting in a way that winds up others and to make them upset. To quote Jeff Sutherland from Scrum: The Art of Doing Twice the Work in Half the Time:
"Don't be an asshole - and don't allow, abet, or accept that behaviour in others."
There has been enough research done to show a good work-life balance is more beneficial for individual productivity than the amount of time they spend on a task. It goes back to the idea of a happy worker. Happiness is a precursor to productivity and success, not vice-versa.
While you are working (including planning) you are making decisions, and decisions take energy. Again, research has shown that our ability to make clear good decisions degrades throughout the day. You also degrade the ability to regulate yourself, leading to more errors. So the more the work, the worse you work.
And as you create errors, you create work. Someone has to fix the error. So the longer you work, the more potential you have to create errors that someone has to fix, meaning you eventually add no value in your effort, and perhaps cost the project.
You are also not a good multi-tasker. Do you know why? Because no one is. Again, all research points to the cost of multi-tasking. Every time you switch tasks you lose time due to switching your brain from one task to another. That time is never zero. You also cannot physically do two things at once. People multi-task because they are distracted and they have trouble controlling their impulse to do other activities. They cannot control their distraction. Be aware and focus on one task till it is done.
Finally, if your team requires people to overwork to deliver the project something has gone wrong - something is not working as it should. If the team is jumping from one crisis to the next then they are not improving. No one cares how many hours someone worked to deliver something. The key is how quickly it was delivered and its quality.
Scrum defines two roles within the team:
- The Product Owner decides what the work should be. He or she owns the Backlog, what's on it, and most importantly, what order it's in.
- The Scrum Master keeps the team working, solves blockages, and keeps the rhythm going. Their job is to help the team work as best they can.
Notice there is no Project Manager, no leader, no boss. Teams are autonomous and self-organising. If there is a problem in the team they collectively solve it. Deal with problem team members. Reflect and be honest. Have integrity and honesty. Get the work done and support the team.
Another key concept of working in a Scrum team is that everyone knows everything. No hiding information or keeping problems to yourself. It helps no-one. This is why Scrum teams are small - everyone can talk to each other. Also, the rhythm of meetings means everyone should be aware of what is happening.
Scrum also avoids handing off tasks between teams. On a handoff information is lost. The team knows everything that is happening, the work that needs done, and how to do it. Everyone knows everything.
How do we run a Scrum team?
- Produce an initial product backlog.
- Prioritise the backlog.
- Do an initial estimate of the work.
- Decide on what work to do in the next Sprint - Sprint Planning.
- Every day have a daily stand-up - Daily Scrum.
- End of the Sprint, review - Sprint Review.
- Analyse how the team has worked and modify based on outcome of review: update estimates, add new tasks, learn lessons, etc. - Sprint Retrospective.
- Repeat 4-7 until product is delivered to the customer.
Scrum uses a Plan, Do, Check, Act (PDCA) Cycle:
Unlike a Waterfall approach, Scrum refines the plan throughout the project. Every iteration - every sprint - planning is done in just enough detail to deliver the next increment. We don't set the plan in great detail because we don't know what could happen.
When producing and managing a backlog there are some points to consider:
- Focus on delivering what is valuable - deliver what people actually want or need. Do not deliver everything you can possibly come up with. Do not add unnecessary features. If you tie up resources in features and tasks that don't deliver value that is a waste.
- Everything cannot be a priority. If everything is a priority, nothing is.
- Time is limited, as is the team's work time. Treat time as finite.
Plan your sprint based on what work is ready. When deciding to do a task you need to ask two questions:
- Is the task ready to be done (e.g., nothing is blocking the task)? You need a Definition of Ready.
- How will you determine when the task is complete? You need a Definition of Done.
For a Definition of Ready, we can use the INVEST Criteria:
- Independent - the task can be completed on its own. It should not be dependent on another task.
- Negotiable - until it is being done a task can be changed.
- Valuable - it delivers value to the customer, user, or stakeholder.
- Estimable - you can size the task.
- Small - task is small enough to estimate and plan. If not, break it down into smaller tasks.
- Testable - the task has a test it will need to pass to determine if it is complete. Write the test before you start the task.
For a Definition of Done we use the testable condition.
To decide what tasks to do, you select from the prioritised product backlog. The Product Owner needs to answer the following questions:
- What tasks will provide the biggest impact?
- What tasks are most important to the customer?
- What tasks can make the most money?
- What tasks are easiest to do?
Remember you will be constantly reviewing, so the priority of tasks will change. That is the point of Scrum - you are constantly reviewing your plan.
Basically, do the work. Try and complete the tasks in the Sprint Plan. Work is only completed when it is Done. Done means you have a complete, deliverable product that can be used by the customer. Work that is half-done is not done. You expended resources, effort and time and not produced a deliverable product. That is the aim of a Sprint.
At the end of the Sprint you need to be able to demo the product to the customer. Demo or die.
Every day at the same time the team stands up and updates on where they are in the project. Everyone answers the following questions:
- What did you do yesterday to help the team finish the Sprint?
- What will you do today to help the team finish the Sprint?
- What obstacles are getting in the team's way?
The daily standup builds routine. It builds a rhythm for the team. It is essential to the team's wellbeing that everyone gets together at the same time every day. Missing these meetings, being late for these meetings, or otherwise being disruptive does not help the team.
At the end of the Sprint the team counts the number of points completed that iteration: the team velocity. You need to know and measure your velocity, but doing it once is not enough. It will take a number of Sprints to get an understanding of how fast the team can work.
Once you know your team velocity, you can perform one of two estimates:
- delivery = velocity * time of project. In other words, if you have a deadline and a velocity, you can estimate how much you can deliver.
- date = work / velocity. In other words, if you have a set of features and work to deliver, you know when you can deliver it.
Typically, the first measure is what we care about. We have a deadline and we deliver to it. As we are delivering value first, we have given the customer what they need.
If the deadline cannot be met, ask the following questions:
- Is there anything we can do differently to speed things up?
- Can we offload some Backlog items?
- Can we not do some things?
The team then undertakes a retrospective as described above to improve how the team works.
To summarise the Scrum Framework see below:
- Pick a team. You need four people for your team.
- Define a Code of Conduct. See Unit 10a for assistance.
- Pick a Product Owner.
- Pick a Scrum Master.
- Create and prioritize a Product Backlog.
- Refine and estimate the Product Backlog.
- Schedule Sprint planning session.
- Schedule daily stand-up.
- Schedule Sprint review.
- Schedule Sprint retrospective.
Your team should work on everything together. Work on the labs together. Study the material together. Work on the coursework together. By this, you will increase the overall capability of the team above what the individuals can do by helping each other. Remember, you are a team.
To illustrate how Scrum ideals (and agile principles in general) can work, watch the following video by Spotify:
- Do we really need to meet once a day?
It would be good if you could, but in reality three times a week will probably be enough. You can use whatever technology you like such as Skype or similar to communicate. The point is to meet at the same time each day for no more than 10-15 minutes.
- We cannot find a time to all meet. What do we do?
You need to commit to your team and your learning. You need to see it as a priority. But life does get in the way. Talk to a member of the module team to discuss how to resolve.
- How will Scrum be used by the teaching team?
In two ways:
- The coursework is assessed at two points in the module but as a group you should break the work into smaller 2-3 week Sprints.
- Each week at least one member of your team (preferably all) should attend the Virtual Office Hour detailed on Moodle. During this time the group will have a 10-minute standup with a member of the teaching team to check how everything is progressing.
- Does my team have to work on the lab material together?
You should attempt the labs yourself but as part of a team you should support each other in your learning and share knowledge with other team members.
- We have a problem with a team member. What should we do?
Identify quickly and resolve by addressing with the team member. If there is a still a problem bring it up with a member of the teaching team. If you have fallen out then ask yourselves if it can be solved. Try and be professional and curtious as a team.
- Can we sack a team member?
We'd prefer not to, but if this is necessary bring it up with a member of the teaching team at the Virtual Office Hour.
Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland.
This is a good read in general. It will step you through building a Scrum team and process, why Scrum uses the mechanisms it does, and provides an overview of how Scrum operates. I highly recommend reading the whole book, but at least accessing the appendix on Implementing Scrum - How to Begin.