Agile SOA - Logical Architecture

To realize the goals described in above post, the following layered logical architecture is employed:

• presentation layer: typically, it is a Web GUI application. Presentation layer invokes Web Services provided by business process layer.
  
• business process layer: It provides Web Service for Presentation layer or direct visit from other systems. It orchestrate the business logic among different system, organization, or different roles. It is built to support change - different application has different business model and logic; and business model and logic change over time.
• service bus routing layer: It routes service requests to the right business process; it features content based routing or dynamic routing.
• service bus data transformation layer: to translate between internal language, described as information model, to external system language.
  
• common service layer: manage internal business entities that are stored in the persistence layer. The following paradigms are used :
• =>1) Common operations: an XQuery based reading operation, an entire XML write operation (course-grained)
• => 2) Specific operations: fine-grained write operations can be added only when performance could be optimized - update certain attributes of a XML document - or a very complex writing to many XML documents. (Note: This is different from traditional approach, which provides setter/getter operation for each attributes of the object). Realization technology: EJB. Web Service is not the first candidate due to performance concern - the invoker is enterprise service bus or the business process layer, which are system internal invocations.
  
• => XQuery Registry could be used to improve development efficiency. Developers get used to getOrderByOrderId("oid") style way to search the database. This new proposed XQuery based read operations requires the user to compose XQuery string directly. XQuery Registry may ease the pain of composing XQuery all the time - it registeres all XQuery string to a central repository and assign a alias for each string. Then a XQuery function extentsion ("agile-soa" as the example namespace) could be used to refer to the the XQuery String directly.
  

• integration layer: integrate with external application - other systems in the same enterprise, or partner's systems.
  
• and persistence layer: typically, it is an XML database.
  

Above logic components could be realized by different technologies:

In below example part, I will use the Oracle based platform. (more tom come)

Thinking in Agile SOA

Problem Domain:

• Complex Business Logic (such as telecom voice&data service purchasing, order handling and activation)
• Complex Enterprise Application Integration (such as integrating with the Online Store, other sales channel system, Order Handling System, Factory/Shipment System, and customer database)
• Complex External System Integration (such as connecting to 10+ telecom carrier's systems to activate services; each operator has it's own business process and interface specification)
  
• High Available solution
  
• Medium Level throughout and performance (If you're looking for high capacity, low latency communication such as service orchestration in SIP, Parlay/X, etc, this discussion may not fit)

Goal of Agile SOA:

• Fast Time To Market
• Release core features in the first launch (you always have a time crunch in today's IT project), while have a extensible framework for further business need
• Enabler small team development, thus to simplifying recruiting, communication, and possibility of applying agile-development or eXtreme Programming. - Work with the team you trust while also reduce the cost.

Is it possible to achieve my goal? So many contradicting statements? Wants to realize a complex project via a small team? and I want it cheap? and fast? So here is the thinking behind this ambition:

• Agile/XP development method should be adopted and team-size must small. Given my experience in another 60man-year development & integration project, the cascade approach did it's job in terms of fulfilling the business requirement, but cumbersome (hard to change), heavy communication required among teams, and thus costly. Agile/XP method will help me cut the effort by changing the way of communication - note, not by reducing the communication but vise versa. Why I need 100+ pages of design document per subsystem and 30+ use case and sequence diagrams per subsystem System Engineering team to development team? They're difficult for design team to fully follow (even dangerous is that they may be ignored by some developers, when I have a big team (all staffed by sourcing team based on high level "resource profile") , in a very unprofessional way.

• Development tools today is different. I want to kill significant amount of POJO and EJB programming. I can lower this type of cost by sourcing it to low cost regions, however, is there any smarter way to avoid it? My tool box here is XML, XML and XML. It has such a good support today, and have XML used in presentation layer, enterprise service bus layer, BPEL layer and persistence (database) layer.
  

• And more important, align tools/technology in all layers of development. So I can save cost by synergizing the technology - no transformation in different layer. Look at the normal/today/(or yesterday's, you may say, if you are already a precursor in this area): RDBMS in database layer - DB Designer is very important to deal with big number of tables; JDO/Hibernate Mapping layer - a large number of Java Bean/configuration there; EJB layer - lots of Java Bean again; Now, lets do some SOA - Enterprise Service Bus and BPEL process Manager will kick in here; Presentation (WEB GUI) is a must - so my developer has its java beans all of the places as the "Model" of the MVC view. The normal transition path is: RDBMS Table -> JDO java beans -> EJB -> BPEL or Proxy Service in ESB -> Java Bean in presentation layer -> JSP/HTML; I want to make it as XML DB -> (an integration tool is missing in my mind as of now) -> BPEL or Proxy Service in ESB layer (they process XML directly) -> XML based JSP/HTML page rendering.

• Information Model is a key to success. Assuming you're new to this information model concept, please refer to TM Forum Shared Information Mode (An information model used in telecom management industry)

• Today's new toolsets blurs the boundary of what Solution Architect (System Engineering or System Architect) team can do and what software design and developers can do. The idea is that Solution Architect should know, or should lean, to use those tools to express the main logic of the system. And on the other side, the design team will use the output directly to detail it. (This is not a new idea, UML based automatic code generation and reverse-engineering were there for a long time...But I haven't see any serious implementation is a real project. Forgive my ignorance here if you did a lot already)
  
  In my terminology, Solution Architect focuses on clarifying the business requirements with customers, discussing business processes to be automated or improved, enact inter-connection specification between the system to be built and other Customer's own systems or partners' systems. I.e., Solution Architect talks details about the business language with customer with very high level mentioning about software architecture and site rollout (servers, rack layout, network connection and etc). And the software design and development teams focus more on the realization and they're are main speaker of EJB, JMS Queue, WSDL and etc.
  
  However, new sets of tools enables Solution Architect, who knows requirements and business process best, gain more control in terms of the realization - to make sure the realization is what he needs. (There was a famous cartoon about this...a link to be shared later). Here are several examples: BPEL development environments is intuitive for the Solution Architect to model what he/she needs instead of Software Design document and UC/Call flow documents; Proxy Service (Such as Oracle/BEA Aqualogic Service Bus) enables Solution Architect to see the routing policy and where are the transformation logic is easily; business rule engine, such as iLog, has the same selling point of enabling even marketing people to change the business rules.
  
  
  
• Sometime, Fuzzy is good. To launch core features first and do the details later, I need to make the design "fuzzy". Here is an example to explain my thought. Let's say, I want to sell my voice plan to a new customer, so I need to include such tariff here " Customer will have 500min per month; Any minutes beyond the quota, customer will be charged 0.1$ per minute during business hours; free during week ends and night; night means from 9:00pm to 5:am the next morning". To cater the need of the purchasing business process, system can model above info as text message and call it "tariff description". In database, it is one column; in java bean, there is one attribute for it; and in presentation layer, it is simply a text box. However, in later phase of my project, I want more fine granularity control, my charging module (actual term is "rating" for insiders) need to understand the logic and calculate the bill accordingly. Then the "text message" is not enough at all and it will be presented in a structured way in all layers, from database, to JDO, to EJB, to Business Process, and to Presentation layer. My new idea is to have a fuzzy design in very first phase, while it can be grown to the fine-control with minimum impact to the work has been done. So I can win the deal by meet the insane launch date while still keep my overall price low.
  

• Technology Neutral Architecture
  
  I could not get even 1 dime by reselling ORACLE/IBM/BEA product, and you don't want to be bound to one tech platform too. So the "thinking" must be able to be realized in a tech-neutral way. The method I will use is:
  
  • Describe the logical architecture in a tech neutral way. So you will not see any big name here
  • But I will use commercial software to realize the reference architecture unless you volunteer to do it based on other platform, such as open source. If there are gaps between the open source platform and commercial platform, you need to spend money there.