(generated by: Midjourney v5, prompted by: Bert Nieves)
Philly, NYC and New Jersey - Attitude & Architecture
I started my software career in the Northeastern US, primarily in New Jersey, Philadelphia and New York. Quickly launched a consulting career after completing undergrad studies at Rutgers University and never looked back. As all of you have probably heard or directly experienced, it can be quite interesting working in North Jersey, NYC (and Philly) area. There’s definitely a rough edge to the people and to the cities and to the work, you quickly learn to be a fighter, learn how to defend positions and advocate for yourself and your teams.
What’s less talked about in startup and Silicon valley dominated engineering circles, is how much great software and technology has been (and still is) built in the Northeast. While Xerox PARC was prototyping the technologies that will eventually launch the computer industry and Silicon Valley including the invention of laser printing, Ethernet, the modern personal computer, graphical user interfaces (GUIs) and the desktop paradigm, object-oriented programming (OOP), the computer mouse, and VLSI design for semiconductors (PARC - Wikipedia), on the East coast, Bell Labs was doing the same in Murray Hill, New Jersey, leading advances in the laser, the photovoltaic cell, the charge-coupled device (CCD), information theory, Unix , and languages such as C and C++ (Bell labs - Wikipedia).
Large, complex software systems were required and enabled the growth and sustainment of the financial sector rooted in NYC, the huge pharmaceutical and early biotech sector in the NJ and Philadelphia region and the great media companies of our age, primarily headquartered in NYC, NJ and Philadelphia during that era. I worked in many of these companies during my consulting years from 1995 - 2012 and worked deep in the trenches of building software across all these industries — there was one common thread — well maybe two, Attitude and Architecture.
Software Architecture in a Pre-Agile World
Software architecture has always held a special place in my heart throughout my entire 25+ year career in software development. I’ve worked with some incredible software architects, and each one has left a deep impression upon me and shaped the way I design and architect systems.
Probably more of a function of the times, then the region - Incubations, Architecture, Analysis and Modeling ruled the first phases of most software projects. I was first introduced to software architecture and modeling by an architect named Greg Slimski. Greg was a coworker of a great mentor of mine named Raymond Johnston who brought me in to my first real post-undergrad software job at General Instrument (GI), an early innovator in cable and satellite TV and network headend systems including building the early versions of the modern day cable box. GI also innovated in early video on demand systems that eventually led to the streaming video systems we all use today, this division was eventually sold to Motorola.
I remember Greg drawing these funny looking clouds on his computer, and he was using these clouds to explain software concepts in our meetings - I was perplexed, confused and intrigued. He was using the Booch method, by Grady Booch, to build out a class diagram. After a couple of meetings and my initial research into the paradigm - I was hooked. The use of abstract ‘Booch clouds’ to logically represent and model concepts in a software system - was a natural complement to the Object Oriented Programming (OOP) classes I took in college.
Booch methodology - Class Diagram
Also, once I found out how much money software architects generally make, it was a no-brainer. After leaving GI, I decided to pursue contract/consulting work and I was committed to becoming a top-notch architect. I purchased several tomes on OOP, OOD and OOA and I carried them everywhere with me, to all my future consulting gigs. From them on, I realized I could differentiate myself and my fledging consultancy as an architect, and not just as a coder.
Greg, if you read this, I know I used to give you a hard time about drawing those Booch clouds and not doing much coding back then - but I secretly wanted to be you, an architect, and this awakening to the wild side of software analysis, design and architecture fueled my launch into a successful consulting career.
Thanks brother !
(image credit: Amazon)
(image credit: Amazon)
Booch methodology was eventually superseded/integrated by the Unified Modeling Language (UML) - this is where I was introduced to the UML Sequence Diagram (Ivar Jacobson style) - which is arguably the most popular architecture diagram you see today. As can be seen from the tweet below, there were several independent modeling standards at play during this time, with significant complementary and/or duplicate approaches to modeling both the static aspects and dynamic aspects of software systems. Eventually UML became the standard
Software Architecture in a (Fr)Agile World
Silicon valley software startups were spinning up at a frenzied rate in Northern California and other regions, these startups were seeking more optimized approaches at building software prototypes and MVPs (Minimal Viable Products) to the market to outflank the larger more established companies in the market. Most started adopting a new hyper-iterative approach to the entire software development lifecycle named Agile immortalized in a publication titled - Agile Manifesto.
Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.
through early and continuous delivery
of valuable software.
Welcome changing requirements, even late in
development. Agile processes harness change for
the customer's competitive advantage.
development. Agile processes harness change for
the customer's competitive advantage.
Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
couple of weeks to a couple of months, with a
preference to the shorter timescale.
Business people and developers must work
together daily throughout the project.
together daily throughout the project.
Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.
Give them the environment and support they need,
and trust them to get the job done.
The most efficient and effective method of
conveying information to and within a development
team is face-to-face conversation.
conveying information to and within a development
team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
Continuous attention to technical excellence
and good design enhances agility.
and good design enhances agility.
Simplicity--the art of maximizing the amount
of work not done--is essential.
of work not done--is essential.
“The best architectures, requirements, and designs
emerge from self-organizing teams.”
emerge from self-organizing teams.”
At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.
to become more effective, then tunes and adjusts
its behavior accordingly.
(generated by: Midjourney v5, prompted by: Bert Nieves)
Agile methodology promotes a set of principles that deemphasize extensive specification and architecture phases during the initial stages of software life cycles. Instead, it encourages frequent collaboration and swift iterations of features or system slices, implemented in a minimal yet end-to-end manner. This approach shifts an architect's mindset towards what I term - "Just-In-Time Architecture."
Startups and small companies began to develop methods to distribute the risk and cost of comprehensive specification and design phases of software projects, which were primarily employed by larger, established corporations and industries with well-known and stable markets. Startups and small SaaS companies grapple with significant uncertainties concerning product requirements and optimal market fit. Consequently, they must distribute and balance that risk and cost across multiple iterations.
This necessity has given rise to PRE-ALPHA, ALPHA, and seemingly perpetual BETAs in numerous SaaS startups. It's an evolutionary response to the daily existential risks they face, such as cash flow and capital availability. Pursuing an exhaustive design and architecture based on flawed or incomplete requirements and specifications could be disastrous, as startups might exhaust their capital and runway before delivering a single feature. Agile methodology addresses this issue by defining essential roles and processes that help mitigate the risk of inadequate (or unknown) requirements. It involves the requirements owner as the Product Owner and makes them a part of the process. Agile also establishes a short, iterative Sprint process that promotes incremental delivery of features, offering the market and customers manageable samples that facilitate feedback (or engagement tracking). This enables the product team to pivot sprint by sprint based on customer or market signals. Gone are the days of tossing 100-200 page functional specification documents "over the wall" to engineering and waiting six months for the first demo, by which time the market may have changed or the feature rendered obsolete or detrimental to the product.
Proof of concepts (POCs), incubations, and spikes have become instrumental in the Agile landscape. These activities empower teams to establish the "line of sight" necessary to confidently tackle complex feature requests and develop them in a risk-mitigated manner.
While I held fast to my UML and architecture foundations, I recognized the immense value in certain aspects of the Agile approach. Eventually, I arrived at a pragmatic intermediate space where I advocate for and schedule an architecture phase more aligned with Agile methods and principles. By integrating principles deeply rooted in OOA/OOD/OOP along with UML and incorporating concepts and activities from other methodologies, such as Domain-Driven Design (DDD), as well as a substantial emphasis on proof of concepts (POCs), I have developed an approach I call "Narrative-Based Design."
Narrative Based Design - An Agile SDLC
As the name suggests, it starts with a short, dense and concise narrative which serves as the foundation of the entire process and subsequent steps.
Here are the primary steps/deliverables of Narrative Based Design, I will deep dive into each step, in the second part of this series.
A System Narrative - A native language summary of the system, in the U.S. this would be an English paragraph (or a couple of sentences) providing a concise summary and description of the system - generally provided by the Product Owner or Product Team.
An Actor List - what users (or systems) will use this product and as what roles will they be using the system (as an administrator? a user? a supervisor? etc)
A UML Use Case Landscape or an Epic Landscape - “The Bubble Chart” - this diagram models the actors (users) defined above along with their corresponding use cases (and/or Epics) which are represented by UML use case ovals (“bubbles”). System boundaries (bounding rectangles) help provide context in these diagrams.
A Logical Block Diagram (or “HLA” - High Level Architecture”) - this diagram provides a host of technical and non-technical stakeholders a simplified birds-eye visual model, describing the overall system (or feature) as a set of blocks, each defining a major subsystem and/or feature, along with how the blocks relate with one another.
A UML Static Structure Diagram (or Logical Class Diagram) - this diagram provides a static view of the important logical entities of the system. No technology or implementation assumptions are made here - just a set of logical entities modeled by the UML class, enumeration and interface symbols and lines connecting the entities describing the nature of the relationship. This often overlooked diagram also helps clarify the language and vocabulary that different stakeholders use when describing similar or the same concepts — a very important step that can help prevent a ton of downstream misunderstandings and lost work.
THIS IS NOT A DATA MODEL but can serve as the foundation of most (if not all) of the information models required including logical and physical data models, JSON and XML documents and schemas, etc.A set of UML Sequence Diagrams (or Activity Diagrams) - this diagram provides an excellent reference and helps synchronize the team around important dynamic or workflow based use cases in the system.
(image: UML static structure diagram - blurred)
These steps help achieve clarity early in the project cycle, ideally before any agile sprints begin - or during “Sprint Zero”, benefits include…
CLARIFY REQUIREMENTS - #1, #2, #3 and #4 (above) helps align the product vision with the high-level architectural model that the engineering team(s) has produced based on their interpretation of the product vision.
CLARIFY SCOPE & SCHEDULE - I can now use #1, #2, #3 and #4 to align scoping around MVP, ALPHA/BETA and/or PHASE I/II/III milestones - this can help the project managers and program management as well - when reporting to executives or founders.
CLARIFY DESIGN APPROACH - Steps #4, #5 and #6 help clarify the architectural guardrails, system boundaries and both the static and dynamic aspects of the system. The architects can now more effectively communicate and align on approach and implement the system.
In the next part of the Agile Architecture Series, we will cover each of the steps and how to incorporate an agile version of each during sprint-based development cycles.