Scrum Log Jeff Sutherland: Story Points: Why are they better than hours?

Scrum Log Jeff Sutherland: Story Points: Why are they better than hours?: Time to update this blog item as interesting new research has become available. The Standish Group has updated their findings on project...

Scrum: In a large enterprise environment can the scrum sprint/iterations synchronized?

I have a simple guideline for this

Leave it to the teams - unsynchronized sprints

• Simple products with single teams working on it
• If you have a different independent products which rarely or doesn’t communicate with each other.

Team will be more creative and it gives more flexibility. Because of all these unsynchronized releases there will be lot of churn so the management & PMO should use a lot of common sense in dealing with such cycles.

Use synchronized sprints

• If you have a big product, with multiple teams working on it
•  If you a product portfolio of products/applications which communicates with each other
•  If you want to synchronize the release because of billing, reporting or any other similar reasons

  

Synchronization helps in large organization where we have to worry about lot of complexities. They will have a common reporting tool which can provide the correct view across all the teams. This helps in billing and other administrative jobs also.

In my company we have multiple teams working in a multi-billion USD product suite. We have a two week sprints which is synchronized across the different teams. There are many teams who have a 1 week sprint also.
e.g. – Assume the organization is in Sprint 03, team A is in two week sprint & Team B has one week sprints
So for the sprint team A will have only one sprint - Sprint 03
Team B will have two sprints - Sprint 03(a) & sprint 03( b).
After the two weeks, both teams will jump to Sprint 04.

How about common resources (people/machines)?

When the teams are in the synchronized mode, they may need some common resources (people or machines). How can we make them available for all the teams when they are synchronized? I have heard this question few times. Usually the planning for the sprint happens during the release planning but what is preventing them from discussing the need for common resources ahead of that? During the current sprint itself team can set aside few hours to identify the resources required for next sprint. The Scrum master should ensure that he communicates the need with other managers and scrum masters and make them available to the team at the right time. If due to any reason the shared resources are not available then it should be informed to PO. The necessary priority changes should be made in the product backlog of the team so that team can work on the next item.

I believe companies shouldn't over synchronize to exact date & time also. There may be people who are in multiple teams - Product owners and scrum masters. Ideally a great PO & SM can serve only one team but still we can see them serving more than one teams. Such “shared” people would miss on important meetings.  When I was the scrum master of two teams, I changed the sprint review dates to the next day. So on one day I could attend the review of first team and the very next day the second teams. On Tuesday I had the review of my first team like other teams in company and on Thursday I had my second teams review. It worked perfectly. These small changes in date and time allowed the little flexibility needed to accommodate the important stakeholders.

Types of Scrum Master; Do we need someone with IT background as a Scrum Master?

Generally there are 4 types of Scrum master

1.Technical - Battle hardened IT engineer who are promoted as SM

2.ProjectManager - SM : Traditional Project Manager who transitioned to SM ( really !!! ). This is a very tricky situation. The traditional PM has to do a lot of un-learning before he/she can become an SM. The PM shouldn’t be made an SM of the same team which he/she was managing.

3.Leader SM - The true leaders SM who love people and help them transform, They may or may not have technical or scrum knowledge. They help in the transition or improvement with their influence and leadership qualities. At any day I would like to work with such thought leaders

4.Scrum Process zealot - SM : Someone who has received a lot of SM training, who follows the rules very religiously to an extend that he/she never allows any innovation (Change) around these. We need such SM’s when the team is new or when going through a lot of volatility. I have personally seen one of my managers helping the team in chaos with his tough rules. Once the team stabilizes such zealots should be transitioned/influenced to exhibit other traits.

We need all these types of SM on different projects depending on their maturity. Over emphasizing any particular trait will not help. Generally I have seen that the best technical person in the team is promoted (transition will be a better word – in agile we cannot promote because all are at the same level ). I am personal example of this (Now you know that I am the best techie. I hate to break this news so openly). I have personally seen that initially it works but the team will still look upto these techie SM for solutions. It was very frustrating for me to hear the pleas for help for providing those simple solutions which anyone can find easily by googling ( or yahoo or bing – I am impartial). There was many ways I made my team think and self-organize. But I had to put additional efforts for this. My effort would have been much easier if I was not a techie SM. Even now sometimes I hear the SOS call for my technical skills. My technical skills made me understand many discussions which happened in the team &I was able to question/influence many decisions. The biggest challenge was to manage myself rather than the team. Whenever I hear the discussion about any technical topic which was dear to my heart, it starts pumping more blood & my brain emits the hormones to my tongue to produce the necessary noise with the information ( oops I have to learn my Bio again , which reminds me that I got 92 % in HSC for biology WOW). Like a project PM I also believe that a techie shouldn’t be made an SM of the same team (especially if he/she was a tech lead). Technical SM should be helping the team members with his suggestions only when they ask and not push his technical knowledge or ideas. Use your brain, don’t answer all the questions guide them so that they can find the answers themselves. Teach you team fishing; not give them fish.

I am attending Agile India 2012

http://betterconf.com/alpha/agile2012.in/#myschedule:s7,s12,s24,s27,s30,s36,s48,s50,s51,s54,s64,s77,s81,s90,s93,s108,s123,s126,s139,s142,s146

Scrum quality – Teams not delivering high quality stories; testing not complete during sprints

You had a sprint demo and it failed miserably. A lot of stories were not fully tested. QA team is complaining that they don’t have any work during the first part of the sprint and they are overloaded during the last few days of the sprint. This looks very familiar.

For months we will work casually and during the last two months of the release there will be lots of last minute changes and bug fixes. It will be a virtual hell with late night hours. People will be coming early and going late (Many don’t even go home, they stay at office). Now this looks familiar. We have seen this in many projects. Scrum promised that such thing won’t happen again but still we can see such mini waterfall in our sprints. If you are seeing such issues, you should have a retrospective immediately.

A scrum team should be cross-functional. I will start with the following questions

• Why can’t the developer’s cross verify each other’s work?

• If testing is the bottleneck why can’t everyone help by testing few stories?

• Why does the QA team have to wait till the last week for testing?

• What is preventing the team from generating the build early?

• Why is the number of bugs increasing - are the unit test cases weak, are the acceptance criteria for the story weak?

• Is this a skill problem? Does the team require technical training?



Based on the answers you will get a nice perspective of the problem.

There are some suggestions also from my side to prevent this

• Implement XP practices like pair-programming, TDD etc. this will improve the quality (less number of bugs).

• No hands-off between dev. & QA (in fact there shouldn’t be such division itself).

• Make testers (dev. or QA) test the code in dev. machine itself before the build is generated. I have done this successful in my team. The feedback cycle was very small. No more time wasted in creating a bug in bug tracking system, assigning it, explaining it, providing the test data etc.

• Slice down the story to lesser size so that it can be managed & finished early.

• Create a continuous integration/nightly build. If build goes out every night then sure nobody can complain that there isn’t any task for testing.

• Team should swarm over the highest priority story first and only after its completion they should move to the second story.

• There is no dev task & QA task. Everyone should do whatever it takes to accomplish the sprint goals. Developers should test. QA should help dev. create code (they can create test data, set up the environment, automate the test case etc.)

Leveraging scrum for large projects (not large teams); number of scrum teams’ vs. team size

The idea of working in iterations producing potentially shippable software is very powerful. Scrum has some very simple rules to help achieve this. Scrum advocates that team size should be 7 (+/- 2). This works most of cases; almost all the software’s can be developed by teams of this size.

One + One = Two Scrum Team = One Big product team

What if I have to increase the team size? My project is complex and I need at least 12 people to deliver the product. Now this becomes interesting. In some cases I have seen such one big scrum team. Historical data has shown that productivity decreases as the team size increase from the optimum number of 7.

So what should I do?

Split the team into two.

On what basis can I split the team?

You can divide the teams into feature, service or subsystem teams; empower these teams to do their jobs. There could be some additional attributes to PBI to identify this linking. Team can work on such PBI from the common PBI list. So there could be teams working on one single area of the product exclusively. But ensure that they don’t become silos. There should be frequent communication, discussions or meetings with the other team.



What about the Scrum Master & Product Owner?

It’s good if the teams have separate scrum masters & Product owners. But the since the teams are working on the same product; the same scrum master & product owner for the two teams will work. If you have separate scrum masters and product owners; ensure that you have a periodic scrum of scrum for scrum masters and product owners. Scrum masters should work together to remove the impediments. The product owners should work together to groom the product backlog

Can I add more teams?

If your product is more complex, you can add more teams to scale up. But there are certain preconditions for these. As the number of teams increase there will be the need of better communication, coding standards, technical reviews, shared builds, code & document versioning, better application servers etc. For any team to succeed such pre conditions or non-functional requirements are very critical. These non-functional requirements should be given more priority than the function requirements. Tomorrow if you are going to start a project which requires 300 people; according to scrum you will need around 42 teams!!! Imagine the type of noise this number of teams can produce if the proper infrastructure is not set up beforehand. Initially you should start with one or two teams only. This team can work on the non-functional requirements like setting up servers, database, build etc. Ensure that some small function requirement is also done in the initial sprints so as to demonstrate the functionality of non-functional PBI’s. For example if you are setting up Continuous integration & nightly build in the first sprint; you can add a small functionality like creation of login page to the sprint. The sprint demo will be a success if the build can install the application and shows the functionality of the login page thus effectively verifying the build tasks.

How will I coordinate the work?

You can start some of the meetings like

• Daily scrum of scrums – to discuss the impediments and risks. This is a scrum meeting where the team members ( or scrum masters of the respective team members )

• Tech review meeting between the different team’s tech leads or team members. Based on the requirement you can decide on the schedule. These meeting can occur before every sprint to discuss the technical challenges faced by the teams & share the best technical practices.

• Scrum of scrum for product owners to groom the backlog

• Release plan review - the product management team should meet periodically to review the status of the product development & delivery. These discussions can be centered on the release Burndown.

Which product backlog to use? A common product backlog or different backlogs for different teams?

This is a very tricky question. I will list down few scenarios which I am aware of. Experiment and find the best method which works for you.

1. Common main backlog and sub-backlogs for each individual team

there will be a common product backlog, the master backlog of the product. Teams can create the sub PBI’s from the main product story and develop the story further. Each team can individually give story points to their PBI. The mother PBI’s story point will be the sum of all the child story points. If any parent story is dropped all the child stories will be dropped.

A hierarchy of product owners working together and slicing the main backlog

If a new main story is added & prioritized all the sub teams (or the teams required) will start working on the new story. The initial break-up of the story will be done by the product owners of the respective teams. After the initial breakup the product owner, the feature development team (& customers) further develops the story, adding the missing links, estimating etc. Each team will have an idea of their velocity. The team will continue adding the stories from the main backlog until their capacity for the release is used fully. There will be a lot of churn as the development goes on. It will be responsibility of the product owners and scrum masters to ensure that the team backlog reflects the actual priority of the main product backlog. In many cases teams will have to plan their stories based on the delivery of the dependent stories from other teams. Such a separation ensures that the team works on the stories which they have developed.



2. Separate backlog for each team; no centralized backlog

This can be implemented for small projects but for very large projects such an arrangement can be disaster. Product owners will have to ensure that each backlog reflects the priority of the customer. Such an arrangement needs a very extensive communication plan between the product owners, teams & customers.



3. One centralized backlog with PBI in different sprints

This is an ideal case. Stories will be prioritized. Each team will pick they stories from the main backlog and develop it. It will very common for one team to work on the stories story pointed by other teams. Personally I don’t like this idea. This leads to a lot of confusion and last minute knowledge sharing. But in case you cannot avoid there are some good technical practices which can minimize the pain.

a. Ensure that you are following the XP practices like pair programming, Continues integration build, nightly build, Test Driver Development (TDD) & Behavior Driven Development (BDD).

b. Apart from the INVEST principles of user story ensure that you add the High Level Design & Low Level Design document to the stories. This will help a third person to understand the stories in a much better fashion.

Importance of qulaity in Agile world

Software which doesn’t works or which is full of bugs is of no value, whether you develop your code using waterfall or agile it doesn’t matter. One of the fundamental principles of agile world confirms this “Working software over comprehensive documentation”. Agile follows the principle of satisfying the customer through early and continuous delivery of valuable software. This fundamental guideline/principle itself emphasizes the importance of the quality. If I talk in scrum terms all the stakeholders are responsible of quality. The Product Owner creates the story, gets the necessary feedback from customers and discusses the acceptance criteria with the team. When the team starts working on these stories, all these quality/acceptance criteria are verified within the sprint (and after the sprint by the customer). Any deviation is noted at the earliest and necessary actions taken.

Scrum - Performance Reviews/Appraisals for agile team

http://runningagile.com/2008/01/22/review-process-for-agile-team-members/

http://scrum.jeffsutherland.com/2006/11/agile-performance-reviews.html

http://jeffsutherland.com/review_template.html

Accomodate Bug fixing in a scrum sprint- (Scrum or Kanban)

A scrum sprint entirely devoted to bug fixing is very bad. But such situations are not very rare.

Scrum works perfectly with known feature deployment. Bug fixing works better with kanban. Bugs are unknown entity; unless team do some sort of exploration/spike work, they cannot commit on it.

http://www.agileweboperations.com/scrum-vs-kanban.

http://en.wikipedia.org/wiki/Kanban_(development)  

We have adopted a mix of Scrum/Kanban model for fixing bugs.
One of my team is in maintenance. During sprint planning we decide on the priority of the bugs and timebox all the issues based on the teams capacity. During the daily scrum meeting we discuss about the progress made. If the issue is critical, an update is send to the entire team every four hours. If any new issues are reported in between, PO prioritizes that and if it is the highest priority, team stops the work on the current bug and starts fixing the new issue. We encourage multiple people to work on the bugs. Once the top priority bugs are fixed, team moves to the next item in the backlog. If we take more time to fix than the anticipated timebox we inform the PO. Based on the discussion team will continue working on the exiting bug or work on a new higher prioritized bug.

Scrum : Non functional requirements testing in scrum sprint

The NFR’s and technical requirements should be part of the user story. These can be added as constraints to the story or along with the definition of “done”. Most of these NFR’s can be tested with unit test framework or automated test scripts. Sometimes the process cannot be tested directly. Suppose software requires triple encryption of bank transactions or such things then it will be very difficult to check the state in between. You will have to test the results to verify the process. The ticket generation or the account balance after the transaction will give a correct picture. Performance testing can be integrated with Continuous integrations build or nightly builds.

Scrum : How to accommodate the infrastructure tasks at the beginning of the project?

Anu is a new scrum master. After the project kick off meeting on further discussion she realized that for a successful delivery as a pre-requisite the team needs a lot of infrastructure work to be completed. PO wanted the feature development work to start as soon as possible. Team had serious concerns about the delivery if they start the actual work without the required infrastructure.

What can be done in this situation?

I met with Anu and we agreed on the following

Identify the most critical requirements required for the team to start work. Do a casual analysis with the subject matter expert and the team and find the cost (duration + software + hardware costs) of doing (and not doing)

Anu held meetings with the team, PO & other stakeholders. This teamwork produced a wonderful infrastructure backlog and was presented to PO. She also added the dependency to this backlog. Nice to have infrastructure were at lower priority. This infrastructure work was added to the first two sprints. Team was also asked to do a sample application integrating all the infrastructure work as part of sprint backlog. With this sample application team validated the infrastructure configuration. The other infrastructure woke like server to do load testing; performance profiling was added to the middle sprints.

Scrum Sprint duration – which is ideal; 4 weeks or 2 Weeks or 1 Week? Checklist

The CTO heard about SCRUM. He asked the PMO to implement it. We got a big training. Everyone is happy. We know everything about scrum- product backlog, sprint backlog, daily scrum, sprint review, retrospective. Wow so cool. But no one told me the duration of a sprint. How can my trainer not know about this? This seems familiar? There should be many yeses.

There is no definite duration for sprint !!!

Ken Schwaber & Jeff Sutherland started the sprints with 30 days duration. Now majority of the companies prefer 2 week sprints. If you don’t have any idea; start with 2 weeks. After the couple of 2 weeks iterations get together with the team, collect their feedback and decide whether you want to change the sprint duration. Personally I have seen teams with sprint duration of 20 days, 15 days, 2 weeks & even 1 week (I have my two teams running on 1 week).

Checklist for sprint duration change.

Reduce the sprint duration

• Poor estimation (if team complains that they work for 8 hours on the task estimated for 4 hours) 
• Poor quality (lot of bugs & performance issues) 
• Unable to meet the sprint objectives  
• Lot of scope creep 
• Nature of deliverables (if the team is responsible for the delivery of small reports or documents which won’t take more than couple of days) 

When the sprint duration is reduced team will meet more often for planning and review. This will help in better planning, estimation & understanding of sprint objectives. PO will also have a better time managing all the changes in the backlog due to frequent changes in customer priority.

Increase sprint duration

• Product backlog is almost stable  
• The inability of the customers/PO/stakeholders to meet periodically to review the delivery 
• Nature of deliverables (some deliverables need long time to develop, integrate & test)

What is scope creep? How can we prevent it?

I have faced this many times. Due diligently I follow the requirement documents and create software. But on many occasions testers/customers will discover new requirements from the existing ones. They may be doing this with right intention but imagine the waste when we have to continuously redo our work every time. Apart from the rework imagine the chaos it can create. If this creates so many problems, should we completely prevent such changes? Never. Software should be adaptable and extensible enough make such changes without adversely affecting other areas/features.


There should be a mechanism by which we should allow these changes to happen but in a controlled manner. Scrum has a definite answer to this problem.


As long as the "creep" doesn't happens during the sprint its fine. Once the items for a sprint are finalized then nothing else should be added. The sprint demo should happen only on the items which were agreed during sprint planning & only on the conditions of acceptance defined. If anything new is found during actual sprint execution or during sprint demo, then it should be noted down and added to the backlog. PO can prioritize these new changes for the team after discussion with the stakeholders.

SCRUM- Who should write a user story

Traditionally user stories (or requirements) were written by Business analysts. They used to prepare big documents after months of study. It was a herculean task. I used to get such UI/Functional specification documents. I have fixed a lot of bugs because I missed few text in such 1000 + pages document. This is not the only interesting part. Some of the requirements were so weird that I often wondered why I am creating the features which no one is going to use. If I had the option I would have recommended a better option. If the BA’s misunderstood some requirements & customers failed to correct those few words in the epic requirement then we will have a nice situation.

In the agile world the story is different. Product Owners are primarily responsible for user stories. But can anyone else also contribute? Yes. Definitely yes

In actual environment many users write user stories. The first requirement may come from end user. The PO, tech architect, scrum master, BA’s... anyone can update this but ultimately it is the PO who is responsible for the backlog.
User stories should be written in a non -technical manner from the perspective of an end user
e.g.
As an x user I should be able to xyz actions.

This user story will be further sliced. The PO (or story writer) shouldn’t spend months is defining the backlog. After fine tuning the stories to an extent this should be put to review to the scrum team. The entire scrum team should work on these stories to understand it perfectly. Any technical constraints /limitations should be noted down & presented to customer. Scrum emphasizes this team work. It’s better to have 5 brains working together than one. The most important section out of these discussions/workshops will be the condition of acceptance (COA’s). Generally these COA’s are not technical. NFR’s can be added as constraints to these conditions.

SOA-based WCF architecture

The WCF architecture is based on the design principles of service-oriented architecture (SOA).

SOA is a framework that is used to design service-based distributed systems. In an SOA-based system, platform-independent services communicate across networked computers or computer processes.


WCF implements.

• Explicit boundaries WCF services function using defined interfaces to identify the communications that flow outside the boundaries of the service.
• Independent services All WCF services are deployed and managed independently; they are independent of deployment, installation, and version issues. Also, each service interaction is independent of other interactions.
• Schema and contract-based communication WCF services communicate with clients by providing only the schema of the message and not its implementation classes. This helps developers change the service implementation in the future without impacting the clients.
• Policy-based compatibility Compatibility between WCF services and clients at run time is determined using published policies. Policies help separate the description of the service from its implementation details.

The SOA-based WCF architecture contains four layers – Contracts, Service Runtime, Messaging, and Hosting.  


Contracts: The Contracts layer describes the WCF message system.
This is described in terms of

• A Service Contract describes the method signatures of a service. It's defined using programming languages, such as Visual Basic and Visual C# and is distributed as an interface  
• A Data Contract enables .NET types to be expressed in XML. The data contract describes every parameter that makes up every message that a service can create or consume.  The message parameters are defined by XML Schema definition language (XSD) documents, enabling any system that understands XML to process the documents.
• Message Contracts define the structure of SOAP messages exchanged between a service and a client, and they allow you to view or modify messages and allows finer-grained control over parts of the message
• Policies and Bindings : Policies and Bindings stipulate the conditions required to communicate with a service.They define the configuration, such as security levels, required by clients to communicate with a service.

Service Runtime
It describes the runtime behaviors of the service

• Throttling controls how many messages are processed, which can be varied if the demand for the service grows to a preset limit.
• Error behaviour decides the sequence of events when an occurs in service. This prevents the display of too much of error information to outside entities.
• Metadata behavior governs how and whether metadata is made available to the outside world
• Message inspection is the facility to inspect parts of a message
• Instance behavior specifies how many instances of the service can be run (for example, a singleton specifies only one instance to process all messages).
• Transaction behavior enables the rollback of transacted operations if a failure occurs. 
• Dispatch behavior is the control of how a message is processed by the WCF infrastructure.
• Parameter filtering enables preset actions to occur based on filters acting on message headers

Messaging
The Messaging layer contains channels that process messages and operate on messages and message headers.A channel is a component that processes a message in some way, for example, by authenticating a message. A set of channels is also known as a channel stack. Channels operate on messages and message headers.


There are two types of channels: transport channels and protocol channels.
Transport channels read and write messages from the network, and protocol channels implement message processing protocols.Some transports use an encoder to convert messages (which are represented as XML Infosets) to and from the byte stream representation used by the network. Examples of transports are HTTP, named pipes, TCP, and MSMQ. Examples of encodings are XML and optimized binary.


Protocol channels implement message processing protocols, often by reading or writing additional headers to the message. Examples of such protocols include WS-Security and WS-Reliability.

• WS-Security is an implementation of the WS-Security specification enabling security at the message layer.
• The WS-Reliable Messaging channel enables the guarantee of message delivery.
• The encoders present a variety of encodings that can be used to suit the needs of the message.
• The HTTP channel specifies that the HyperText Transport Protocol is used for message delivery.
• The TCP channel similarly specifies the TCP protocol.
• The Transaction Flow channel governs transacted message patterns.
• The Named Pipe channel enables interprocess communication.
• The MSMQ channel enables interoperation with MSMQ applications.

Fig- A typical WCF architectural flow





Hosting and Activation
The Hosting layer describes the ways in which a WCF service can be hosted. WCF services can be hosted using either Windows Activation Services, Internet Information Services, Windows Services, .EXE, or COM+.

•  A service is a program. Like other programs, a service must be run in an executable. This is known as a self-hosted service.
• Services can also be hosted, or run in an executable managed by an external agent, such as IIS or Windows Activation Service (WAS). 
• COM+ components can also be hosted as WCF services.
  

Windows Communication Foundation (WCF) - Glossary

WCF Fundamentals

• Service are applications that wait for clients to communicate with them and respond to that communication. They expose the functionalities to client.
• Client initiate the communication. They consume the service offered by Service.
• Message: A message is a self-contained unit of data that may consist of seveal parts, including a body and headers. Clients & Service  communicate using XML messages.

A single application can act as both a client and a service.


Endpoints
Messages are sent between endpoints. Endpoints are places where messages are sent or received (or both), and they define all the information required for the message exchange. A service exposes one or more application endpoints (as well as zero or more infrastructure endpoints), and the client generates an endpoint that is compatible with one of the service's endpoints.


An endpoint describes in a standard-based way where messages should be sent, how they should be sent, and what the messages should look like. A service can expose this information as metadata that clients can process to generate appropriate WCF clients and communication stacks.









An endpoint consists of these components:

• address Each endpoint has a unique address that helps a client know where it is located on the network.It is specified as a Uniform Resource Identifier (URI). The URI schema part names the transport mechanism to use to reach the address, such as HTTP and TCP. The hierarchical part of the URI contains a unique location whose format is dependent on the transport mechanism.  The client uses this address to send messages to an endpoint. An address can be absolute or relative. If you enter a relative address, WCF uses the absolute base address as the prefix for it. An example of an absolute address is http://localhost:8888/SampleService.
• binding The binding of an endpoint describes the channels used to communicate with the endpoint. A channel is a route through which all messages pass in a WCF architecture. A channel is composed of binding elements that tell clients which transport protocol, encoding, and security configurations should be used to communicate with the endpoint. WCF provides many default bindings such as BasicHttpBinding and NetTcpBinding.
• contract An endpoint's contract informs clients about the type of capabilities or features the endpoint provides. It tells clients the form of the message, what operations they can call, how they can call an operation, and what response message they will receive from the endpoint. A contract is created by applying the  ServiceContract attribute to an interface or class description and applying the OperationContract attribute to each method included as a service operation.

An WCF service is exposed to the world as a collection of endpoints.

• Infrastructure endpoint An endpoint that is exposed by the infrastructure to facilitate functionality that is needed or provided by the service that does not relate to a service contract. For example, a service might have an infrastructure endpoint that provides metadata information.
• Application endpoint An endpoint exposed by the application and that corresponds to a service contract implemented by the application.
• Binding element A binding element represents a particular piece of the binding, such as a transport, an encoding, an implementation of an infrastructure-level protocol (such as WS-ReliableMessaging), or any other component of the communication stack.
• service operation: A service operation is a procedure defined in a service's code that implements the functionality for an operation. This operation is exposed to clients as methods on a WCF client. The method may return a value, and may take an optional number of arguments, or take no arguments, and return no response. For example, an operation that functions as a simple "Hello" can be used as a notification of a client's presence and to begin a series of operations.
• service contract :The service contract ties together multiple related operations into a single functional unit. The contract can define service-level settings, such as the namespace of the service, a corresponding callback contract, and other such settings. In most cases, the contract is defined by creating an interface in the programming language of your choice
• Operation contract: An operation contract defines the parameters and return type of an operation. When creating an interface that defines the service contract, you signify an operation contract by applying the OperationContractAttribute attribute to each method
• Message contract:A message contract describes the format of a message. For example, it declares whether message elements should go in headers versus the body, what level of security should be applied to what elements of the message, and so on.
• Fault contract: A fault contract can be associated with a service operation to denote errors that can be returned to the caller. An operation can have zero or more faults associated with it. These errors are SOAP faults that are modeled as exceptions in the programming model.
• Data contract: The data types a service uses must be described in metadata to enable others to interoperate with the service. The descriptions of the data types are known as the data contract, and the types can be used in any part of a message, for example, as parameters or return types. If the service is using only simple types, there is no need to explicitly use data contracts.

Behaviors

• A behavior is a component that controls various run-time aspects of a service, an endpoint, a particular operation, or a client.
• Behaviors are grouped according to scope.
• Common behaviors affect all endpoints globally, service behaviors affect only service-related aspects, endpoint behaviors affect only endpoint-related properties, and operation-level behaviors affect particular operations. For example, one service behavior is throttling, which specifies how a service reacts when an excess of messages threaten to overwhelm its handling capabilities. An endpoint behavior, on the other hand, controls only aspects relevant to endpoints, such as how and where to find a security credential.

Communication Protocols
One required element of the communication stack is the transport protocol. Messages can be sent over intranets and the Internet using common transports, such as HTTP and TCP. Other transports are included that support communication with Microsoft Message Queuing (MSMQ) applications and nodes on a Peer Networking mesh. More transport mechanisms can be added using the built-in extension points of WCF.


Encoding is specified in communication stack which specifies how any given message is formatted. WCF provides the following encodings:

• Text encoding, an interoperable encoding.
• Message Transmission Optimization Mechanism (MTOM) encoding, which is an interoperable way for efficiently sending unstructured binary data to and from a service.
• Binary encoding for efficient transfer.
  

More encoding mechanisms (for example, a compression encoding) can be added using the built-in extension points of WCF.


Message Patterns
The message exchange patterns used in WCF are either

• One Way: The One Way is when the sender sends a message to the receiver but the sender doesn't expect the receiver to reply to this message, it is something like Fire & Forget.
• Request/Reply: The Request / Reply is when the sender sends a message to the receiver and expects ONLY one reply from the receiver, as an example any web page you request for it in your browser you expect a reply (Response) which is the Page you requested.
• Duplex: The Duplex is when both Sender & Receiver construct a channel between them and everyone of them can send messages to the other at well, it is just like a Phone Call.

Hosting: A service must be hosted in some process. A host is an application that controls the lifetime of the service. Services can be self-hosted or managed by an existing hosting process.


Self-hosted service: A self-hosted service is one that runs within a process application that the developer created. The developer controls its lifetime, sets the properties of the service, opens the service, and closes the service.


metadata

• The metadata of a service describes the characteristics of the service that an external entity needs to understand to communicate with the service. Metadata can be consumed by the ServiceModel Metadata Utility Tool (Svcutil.exe) to generate a WCF client and accompanying configuration that a client application can use to interact with the service.
• The metadata exposed by the service includes XML schema documents, which define the data contract of the service, and WSDL documents, which describe the methods of the service.
• When enabled, metadata for the service is automatically generated by WCF by inspecting the service and its endpoints. To publish metadata from a service, you must explicitly enable the metadata behavior.

WS-* Shorthand for the growing set of Web Service (WS) specifications, such as WS-Security, WS-ReliableMessaging, and so on, that are implemented in WCF.

Windows Communication Foundation - overview

WCF is a framework ( yep another framework :D ). It is a unified programming model for building service oriented applications

 

The WCF architecture uses message-based communication. This involves messages being sent between endpoints generated by either a service or a client.

A service is an application that responds to a request, and a client is an application that initiates a request. In many cases, a single application can act as both a client and a service, depending on the situation.





WCF is implemented primarily as a set of classes on top of the .NET Framework’s Common Language Runtime (CLR). Because it extends their familiar environment, WCF allows .NET developers to build service-oriented applications in a familiar way.


The benefits of WCF

• asynchronous one-way messaging Many applications use asynchronous one-way messaging. For example, web browsers send requests to web servers and wait for replies. WCF supports asynchronous one-way messaging, which provides advanced functionality, reliability, and application responsiveness. It also makes efficient use of available processing power. Asynchronous messaging is the most efficient way of performing the input and output tasks required by a distributed application.
• Support for cross-vendor interoperability, including  security, and transactions.  WCF has interoperability features that were previously spread across different technologies. It communicates using SOAP-based web services (WS-*) and supports Representational State Transfer (REST) architectures, Plan Old XML (POX) messaging systems, and JavaScript Object Notation (JSON) data during WCF runtimes. In addition, you can write custom extensions that enable WCF applications to communicate with applications that require proprietary message encodings.
  WCF allows transactional scopes to flow across multiple applications
  Because WCF adheres to the WS-Security specifications, its default security options range from message-based security to the more traditional transport-centric security model.
• reliability:WCF provides four assurances for reliability for distributed computing – at most once, at least once, exactly once, and in order. An assurance is similar to a guarantee. It contains mechanisms that provide these assurances with little or no modification to the application. As opposed to traditional types of assurances, WCF's assurance mechanisms don't depend on the transport method used.  
• platform consolidation

WCF unifies the following distributed communication technologies


ASP.NET Web Services (ASMX) and the Web Service Enhancements (WSE): The ASMX and WSE technologies provide an interoperable, service-oriented infrastructure and programming model that can easily be integrated into web services or web service clients. They work on HTTP only and can have performance issues but are interoperable because their data is encoded using XML.
Enterprise Services: Enterprise Services is a component-oriented technology that provides transaction integration across multiple objects performing related work in a distributed environment. It minimizes throttling, optimizes pooling of object instances, provides a publish/subscribe mechanism for events, and uses a fast, secure, and platform-integrated transport method. However, Enterprise Services provides poor interoperability, because it is tightly coupled with the infrastructure.
Microsoft Message Queue (MSMQ): MSMQ is a set of objects that provide a durable, volatile, and scalable message queuing system that ensures reliable data transport from one place to the next. It is used to collect a group of messages, send them to a server for processing, and receive a reply from the server. However, MSMQ cannot process corrupted messages efficiently. When a corrupted message is received by the server, it blocks other messages in the queue.
Remoting: Remoting is the application programming interface (API) used by .NET to allow .NET Framework applications to communicate across application domain boundaries. It is a very flexible and extensible model that enables developers to manipulate proxy mechanisms, transports, and the way communication channels function. However, Remoting does not provide interoperability with non-.NET applications.  

• Explicit support for service-oriented development.

explicit boundaries WCF services function using defined interfaces to identify the communications that flow outside the boundaries of the service.
independent services All WCF services are deployed and managed independently; they are independent of deployment, installation, and version issues. Also, each service interaction is independent of other interactions.
schema and contract-based communication WCF services communicate with clients by providing only the schema of the message and not its implementation classes. This helps developers change the service implementation in the future without impacting the clients.
policy-based compatibility Compatibility between WCF services and clients at run time is determined using published policies. Policies help separate the description of the service from its implementation details.


The SOA-based WCF architecture contains four layers – Contracts, Service Runtime, Messaging, and Hosting.  


Contracts: The Contracts layer describes the WCF message system.
They are of two types- Data & Service

• A Service Contract describes the method signatures of a service. It's defined using programming languages, such as Visual Basic and Visual C#.  
• A Data Contract enables .NET types to be expressed in XML.
• Message Contracts define the structure of SOAP messages exchanged between a service and a client, and they allow you to view or modify messages.  
• Policies and Bindings
• Policies and Bindings define the configuration, such as security levels, required by clients to communicate with a service.

A practical experience
Last year i was associated with a project (.Net 2.0) which had both Java & .net modules interacting with each other. More over there were custom adapters/interfaces (in C#) for around 500 external devices which interected with our application. Few devices provided the data for our application and few others were dependant on the data generated by our application. A specific set of meesages were send  to MSMQ for other devices to read. It was a distirbution nightmare. We had web services. custom adapters, MSMQ, remoting almost all the distributed technologies supported by Microsoft in one project. Initially we had a very difficult time managing all these.


We migrated the project to .Net 3.5. To solve this distribution nightmare we adopted WCF. The following factors helped us to choose WCF

• Because WCF can communicate using Web services, interoperability with other platforms that also support SOAP, such as the leading J2EE-based application servers, is straightforward.(We can also configure and extend WCF to communicate with Web services using messages not based on SOAP, for example, simple XML formats like RSS)
• Managing object lifetimes, defining distributed transactions, and other aspects of Enterprise Services are now provided by WCF. They are available to any WCF-based application, which means that the our application can use them with any of the other applications it communicates with.
• The WCF option for queued messaging, built on Message Queuing, allows applications to use persistent queuing without using another set of application programming interfaces.
• Performance is of paramount concern for most businesses. WCF is developed with the goal of being one of the fastest distributed application platform developed by Microsoft. (To allow optimal performance when both parties in a communication are built on WCF, the wire encoding can be used, in this case is an optimized binary version of an XML Information Set. Messages still conform to the data structure of a SOAP message, but their encoding uses a binary representation of that data structure rather than the standard angle-brackets-and-text format of the XML 1.0 text encoding.)
  

Asp.Net – Creating a new custom HTTP Module

An HTTP Module is a .NET class that executes with each and every page request. You can use an HTTP Module to handle any of the HttpApplication events that you can handle in the Global.asax file.

You are already familiar with some of the HTTP Modules like

• FormsAuthenticationModule handles the Forms authentication
• WindowsAuthenticationModule handles the Windows authentication
• SessionStateModule manages the session state of ASP.net application
• OutputCacheModule deals with output caching
• ProfileModule used for interaction with user profiles

Each HTTP Module subscribes to one or more HttpApplication events. For example, when the HttpApplication object raises its AuthenticateRequest event, the FormsAuthenticationModule executes its code to authenticate the current user.

Below I have included a sample class which implements the IHttpModule interface

In the Init event I have created an EventHandler for the PostAuthorizeRequest event of Http Application

namespace AspNet

{

    public class CustomContentModule : IHttpModule

    {

        public void Init(HttpApplication app)

        {

         

            app.PostAuthorizeRequest += new EventHandler(SendRequest);

        }

        public void SendRequest(Object sender, EventArgs e)

        {

            HttpApplication app = (HttpApplication)sender;

            app.Response.Write("Hi content from HTTP Module
Pay my tax also !!!");

        }

        public void Dispose() { }

    }

}

You can use any of the following events of HTTP Application; the events are raised in the following order:

1. BeginRequest
2. AuthenticateRequest
3. PostAuthenticateRequest
4. AuthorizeRequest
5. PostAuthorizeRequest
6. ResolveRequestCache
7. PostResolveRequestCache
   
   

      After the PostResolveRequestCache event and before the PostMapRequestHandler event, an event handler (which is a page that corresponds to the request URL) is created. When a server is running IIS 7.0 in Integrated mode and at least the .NET Framework version 3.0, the MapRequestHandler event is raised. When a server is running IIS 7.0 in Classic mode or an earlier version of IIS, this event cannot be handled.

8. PostMapRequestHandler
9. AcquireRequestState
10. PostAcquireRequestState
11.  PreRequestHandlerExecute

      The event handler is executed.

12. PostRequestHandlerExecute
13. ReleaseRequestState
14. PostReleaseRequestState

      After the PostReleaseRequestState event is raised, any existing response filters will filter the output.

15. UpdateRequestCache
16. PostUpdateRequestCache
17. LogRequest.

      This event is supported in IIS 7.0 Integrated mode and at least the .NET Framework 3.0

 

18. PostLogRequest

      This event is supported IIS 7.0 Integrated mode and at least the .NET Framework 3.0

19. EndRequest

After creating this class add the following entry in Web.config

            <httpModules>

                        <add name="CustomContentModule" type="AspNet.CustomContentModule"/>

            httpModules>

When you excute any page in this application HTTP Runtime will add the content from the HTTP module created by us. You can build on this to create more complex scenarious like custom authitication, generation.processing of data based on the paraametr availalle in the Http Application objects etc.

ASP.net - implementing Asynchronous HTTP handler

Asynchronous programming has its own benefits. It helps in better usage of resources. In ASP.net when a user request for a resource which is to be processed by HTTP Handler a thread is created an allocated to the handler file. The thread will be idle till the file finish it’s processing. So as to minimize this “idle” period using asynchronous programming we can release the thread back to pool after passing the execution to handler file. When the asynchronous handler completes its work, the framework reassigns a thread to the original request and the handler can render content to the browser. For a better understanding of Asynchronous programming in .Net visit: Asynchronous Programming Overview

We can create an asynchronous HTTP handler by implementing the IHttpAsyncHandler interface, which is derived from the IHttpHandler interface. It adds two additional

methods:

• IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData);—Called to start the asynchronous task.
• void EndProcessRequest(IAsyncResult result)—Called when the asynchronous task completes

Create a new class like the following

    public class NewClass : IHttpAsyncHandler

    {

        private HttpContext _context;

        private WebRequest _request;

        public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)

        {

        }

        public void EndProcessRequest(IAsyncResult result)

        {

        }

        public bool IsReusable

        {

            get { return true; }

        }

        public void ProcessRequest(HttpContext context)

        {

            throw new Exception("The ProcessRequest method is not implemented.");

        }

    }

You can use this class to do a variety of tasks.

In the below Code I have used created a clss to read RSS data from web

public class RSSHandler : IHttpAsyncHandler

    {

        private HttpContext _context;

        private WebRequest _request;

        public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)

        {

            // Store context

            _context = context;

            // Initiate call to RSS feed

           

            _request = WebRequest.Create(context.Request.QueryString["rssURL"]);

            return _request.BeginGetResponse(cb, extraData);

        }

        public void EndProcessRequest(IAsyncResult result)

        {

            // Get the RSS feed

            string rss = String.Empty;

            WebResponse response = _request.EndGetResponse(result);

            using (response)

            {

                StreamReader reader = new StreamReader(response.GetResponseStream());

                rss = reader.ReadToEnd();

            }

            _context.Response.Write(rss);

        }

        public bool IsReusable

        {

            get { return true; }

        }

        public void ProcessRequest(HttpContext context)

        {

            throw new Exception("The ProcessRequest method is not implemented.");

        }

This class will read the RSS feed asynchronously.

The BeginProcessRequest() method uses the WebRequest class to request the page that

contains the RSS headlines. It received the URL as QueryString.The WebRequest.BeginGetResponse() method is used to retrieve the remote page asynchronously. When the BeginGetResponse() method completes, the handler’s EndProcessRequest() method is called. This method retrieves the page and renders the contents of the page to the browser.

Before we start using this handler we have to register this handler like the following in web.config

<httpHandlers>

      <add verb="*" path="*.rss" validate="false" type="AspNet.RSSHandler"/>

                  <add verb="*" path="*.xml" validate="false" type="AspNet.RSSHandler"/>

 httpHandlers>

After building and typing the following URL

http://localhost:2141/xyz.rss?rssURL=http://feeds.feedburner.com/blogspot/anKd

http://localhost:2141/sample.xml?rssURL=http://feeds.feedburner.com/blogspot/anKd

I got an output similar to the one shown below

There was no file xyz.rss or sample.xml but since these extensions were mapped to the recently created HTTP handler output was generated by the framework

Related Articles :

1. ASP.net - Http handlers - implementing IHttpHandler
2. ASP.net - Http handlers - Generic Hander