Whatever I have explored and experienced, a Technology/Solution Architect (Microsoft Technologies) or a good Developer as per Industry Standards, needs to be proficient in as many areas as possible as given below(I think I am still lagging in lots of the areas and that’s the confession, but I am trying to cope up as well) –
Some other “Nice To Haves” are knowledge of BPMN, Archimate, DSLs(very essential for implementing Symbolic Programming constructs – some examples of DSL are JACE .NET and Regular Expression Parser – probably the best resource to learn building any kind of DSL is Language Implementation Patterns, written by the ANTLR author,Terence Parr), Cloud Computing(Windows Azure,AWS,Google Cloud etc), NoSQL DBs (Data Modelling Techniques shows the overall idea of how to think and design the NoSQL way while Polyglot Persistence, Big Data and Designing Data Intensive Applications seems to be some other good texts explaining the whole gamut of the intricacies of WHYs and HOWs of the current state of the art Distributed Data Technologies – some nice articles on NoSQL Movement are What is Hadoop? What is MapReduce? What is NoSQL?, An Introduction to NoSQL Patterns, NoSQL DB Patterns, NoSQL DB Patterns and Polyglot Persistence and NoSQL and Big Data Design Patterns), Big Data & Analytics, IoT(Internet Of Things), Artificial Intelligence(including Knowledge Representation, Web & Enterprise Search,Machine Learning,Natural Language Processing, Computer Grahpics & Computer Vision,Computational Advertising,Fuzzy Logic,Deep Neural Networks,Evolutionary Computing etc – a very vast topic in itself but makes possible some of the coolest applications on Earth), GIS(Geographical Information Systems – some good resources for the same can be found here) etc.
– to have a collection of awesome .NET libraries, tools, frameworks and software, visit this Git-Hub repository.
ii) A very basic Thumb Rule to follow while designing any software in general-
Keep your static parts(rarely changing) separated from your dynamic parts(modifiable or extensible parts) – this separation could be at the code or component or system level.If you meticulously analyze any S.O.L.I.D principle or some Design Pattern(including IoC/DI or some Concurrency Pattern) or some Architectural Pattern, the basic underlying design principle followed is to separate out the static parts and the dynamic parts.This ultimately leads to better maintainability,extensibility, re-useability, separation of concern,reliability(being better testable) etc.In-fact, this principle can be extended to design Data Models or to implement maintainable stored procedures as well. By the way, at the application code level, the static parts, if needed, can be separated out into different classes(may be using partial classes – although the partial class approach is very .NET specific) for better modularity.
iii) Big Data, Analytics and IoT(Internet Of Things) –
The above topics are quite booming nowadays and probably will rule the IT world for quite sometime.
iv) AI(Artificial Intelligence) basically consists of –
b) Search & Planning – implemented using some advanced Back-Tracking or Branch and Bound algorithms.
c) Learning(from Data)
– implemented using Machine Learning
(including Natural Language Understanding and Processing
i.e. algorithms inspired from Mother Nature, Granular Computing
, Multi Agent Systems
etc.These concepts are heavily used in Analytics
, Semantic Web Search and Semantic Enterprise Search
, Computer Vision
(although, knowledge of Computer Graphics
here, can provide you an edge),Human-Computer Interaction
, Computational Biology
, Computational Advertising
, Cognitive Computing
, Computational Finance
d) Reasoning(through Data)
– based on concepts of Mathematical Logic(Predicate Logic,Propositional Logic etc) and Fuzzy Logic
-> A Side Note –
But, before digging into the depths of AI, I would suggest to have a solid foundation of Mathematics for Computer Science including Algebra
, Number Theory
, Graph Theory
, Mathematical Logic
, Linear Algebra
, Multivariable Calculus
, Mathematical Programming
, Probability & Statistics, Game Theory
(Game Theory Resources – I
), Auction Theory
etc, Data Structures and Algorithms since most of the practical aspects of real world AI apps needs solid underlying Mathematics which is implemented using some basic/advanced Data Structures and basic/advanced Algorithmic Techniques.Just Enough Math
can be helpful here to some extent. Other resources like Data Science Starter Kit
and Machine Learning Starter Kit
can take you to the long mile.Basically, you need to be a very good Mathematician as well as a very good Algorist which I am still nowhere near to and still struggling 😦 . And ofcourse you need to have good understanding of Distributed Computing if there is a need to implement customized elastic scalability of AI apps(else Cloud is there to save you). Probably, after that, you can go through Carnegie Mellon University courses
to have a gist of capabilities of AI and it’s related branches.A very good resource dealing with almost any thinkable aspect of AI is AI Topics
.Industrial and Engineering Applications of Artificial Intelligence and Expert Systems
seems to be some other promising reads.
v) CEP(Complex Events Processing) & Big Data –
vi) AOP(Aspect Oriented Programming)-
Gurus might suggest that AOP ultimately lowers the performance of the application since most of the AOP frameworks use code interception/injection using some sort of reflection or some similar technique.But one really useful usage scenario for higher regions(SIT,UAT,PROD etc) can be Tracing of method parameter values and method returned values. Sometimes we find that we get some hard to debug issues in higher regions and think that if we could have the traces of all the values that went into some method(s) and values returned from some method(s) very quickly – atleast for SOMETIME – then the issue could have been resolved much quicker.Well this is basically Tracing and we can keep log of such traces by writing some normal tracing module and calling the tracing module api(which will log the traces in some file system or DB or whatever) in each and every method.Yuck! That’s very hectic.Well, don’t worry, be happy.Because there is AOP to the rescue.Just write some nominal code proactively while developing the app code and incorporate AOP in the app and that’s all. Well is it really that’s all??? Well not yet.As mentioned earlier, having AOP in place for the whole life cycle of the running application can degrade the performance.Well, again, don’t worry.There are lots of AOP frameworks in the market which lets you configure to use or not to use AOP(while the app is running) just by setting some config values.So when you need to enable Tracing to debug some issue in higher region, go for AOP else just keep it disabled wherein no code interception/injection will be there and performance will be normal.
By the way, that’s not the only usage of AOP. In general, AOP (sync/async) techniques can be used for any instrumentation stuffs like Web Analytics, Health Monitoring etc. In fact, AOP techniques can be used for implementing Caching, Validation, Authorization, Performance Profiling and Exception Logging as well.
vi) Enterprise Architect, The Awesome tool – Probably the best tool to use to construct any diagram,be it UML or BPMN or some other Architectural diagram, seems to be Sparx System’s Enterprise Architect
.Although there is Visio or Visual Studio Ultimate Edition’s Architectural features(very .NET specific), but when you compare these tools with Enterprise Architect, it seems as if you are comparing an ant with an Elephant.
– Get familiar with Code Review Process(as per Coding Standards in general and specific to Project). If needed download some good Coding Standards document from internet and use that.Get familiar with Static Code Analyzers like Visual Studio InBuilt Tools,NDepend,SONAR,FxCop etc.If possible use Resharper and if needed go for a Reflector as well. Visual Studio 2013 Static Code Analysis in depth
gives you an in-depth analysis of most of the Static Code Analysis techniques available surrounding Visual Studio environment.
Sometimes you may need to review code from a Functional point of view as well(so, understand well, the domain as well) or may need to do application specific code comparison as well(especially, applicable for Re-Engineering Projects). While at other times you may be asked to do optimization based review i.e whether the code/design is optimized with respect to re-usability, extensibility, maintainability, testability, security, performance etc.
Basically, while doing any review look for project best practices like Coding Standard, Architecture, Governance, Automated Build & Deployment, Change/Release Management(Request For Change, Release Notes, Run Book etc), Unit testing, TDD, Cyclometric complexity(CC) report, Check in policies, Environment readiness, Design for scale, if they have done/doing POC to validate or not etc..
– Get hold of Profiling tools(NP .NET Profiler
, Visual Studio Profiler,RedGate’s Ants Profiler etc) to profile application/database memory usage & performance.Learn .NET Application and Database Performance Tuning. For OLTP based systems, TPM-C benchmarks
are quite good.
N.B. – Certain high level solutions related to Performance issues are –
b) Reactive,Parallel and Asynchronous Computing(not necessary that these paradigms needs to applied w.r.t only multi-core machines but rather can be applied in a totally distributed environment using commodity machines as well – High Performance Computing takes the distributed approach using multi-core machines).
c) Hardware Scaling – both Vertical(increasing the RAM size etc) and Horizontal(increasing the number of machines and using a load balancer – this approach can be used for Disaster Recovery as well using Fail-over Clustering)
d) Database Performance Tuning(probably one of the biggest topics with issues related to performance – an ocean of knowledge in itself).
e) Another option (as far as OLAP systems are concerned ), can be, to use an amalgamation of NoSQL DBs(enables super fast data retrieval) and Hadoop(a very high performance computation framework) Ecosystem.
f) And ofcourse, any combination of the above mentioned performance tips.
g) A very .NET specific performance tip is to use Interop Services and Reflection code as less as possible.As far as Reflection is concerned, try to use some caching mechanism alongwith using DynamicMethod or Expression Tree wherever possible.
N.B. -> Some awesome resources w.r.t Performance tuning/testing are –
4)Documentation – Be strong in documentation(HLD and LLD).Get some good HLD(NFRs ,Cross Cutting Concerns,Physical/Deployment View,Logical View,Dynamic view having sequence and activity diagrams,Backout Strategies etc) and LLD samples of large/complex Web apps and large/complex BPM & EAI based Apps from the internet and understand the WHYs and HOWs.Have one HLD and multiple LLDs as per the number of modules in the Application.
Certain things to keep in mind while doing any estimate –
Break the work tasks into as much detail as possible and then give the effort estimate(it’s always easier to give the estimate by following this simple approach).Certain things can be re-used, take those also in consideration.Break up the estimate into different phases of SDLC – e.g. User Stories or Screen Shots or Code Analysis(reverse engineering the code to extract the requirements can also be a scenario – sometimes if you just give the estimate based on going by the screens, you might fall into a big trap – very meticulous and thorough analysis is the key) or all of these,Designing as per new Technologies,Construction and Manual and Coded Unit Testing.Scale(may be out of 5) the complexity of the requirements (if possible try to give the rationale behind the same) and also have a scale(again may be out of 5) for resource competency and productivity and based on that do the estimate.Provide all these inputs in some excel sheet to make it more understandable.If any of these are missing, then please ask the team to prepare the estimate on these detailed lines and prepare the excel sheet accordingly else it’s very difficult for an individual to review an estimate independently.All these guidelines basically makes it easier for a reviewer to know the basis(rationale) behind the estimate and also makes the estimate more justifiable to the client.
Be very much aware about the scope.Some other factors like Code Analysis,Code Refactoring,Code Merging and the whole gamut of Application Lifecycle Management’s associated approval process should also be taken into consideration while doing any estimate.Separating out the must haves and nice to haves can also reduce your estimate by quite an ample amount of effort.
N.B.-> A Random Thought on Estimation ->
Accurate estimation of software development effort is critical in Software Engineering. Underestimates lead to time pressures that may compromise full functional development and thorough testing of software. In contrast, overestimates can result in noncompetitive contract bids and/or over allocation of development resources and personnel.So basically, any Project Estimation is a problem related to Optimization Theory in general and Machine Learning is a great tool to solve most of the real world Optimization problems and Project Estimation is no exception.Agreed that too many input parameters are there for a Project Estimate(alongwith changing requirements) and you need a humongous amount of data to practically predict some estimate for a project using Machine Learning tools but it’s not impossible, as suggested here
(the techniques may not be applicable for all the scenarios – it all depends on the context of estimation). Some good pointers to apply Machine Learning to Software Engineering in general (including Project Estimation), seems to be Machine Learning Applications in Software Engineering
, Advances in Machine Learning Applications in Software Engineering
, Software Engineering with Computational Intelligence
, Computational Intelligence in Software Engineering
– Get acquainted with new UI and Application Frameworks and adapt to them ASAP.If needed, do some PoCs.Also if needed, try to develop Frameworks for custom needs(on your own). Also, all of a sudden, you might be asked to extend custom 3rd party Frameworks of which you might not have any knowledge.The problem with such Frameworks is the Learning Curve and it’s also probable that you won’t get any help from google for such Frameworks.Only the documentation(if any) and probably GOD can save you :). Please assess the associated Risks,Assumptions and Issues here. Some examples of Business Application Frameworks are like CSLA.NET
(Business Development) Activities
– Might need overall solutioning knowledge rather than just knowledge of .NET(or other language) specific technologies to develop the response for an RFP(Request For Proposal). Also, should be acquainted in preparing presentations for RFPs and making the stakeholders understand the same.This might need to get involved in Pre-Sales activities alongwith the Business Analysts.Also, may need to understand the domain in totality and develop the Business Architecture
alongwith developing the Information Architecture(Data Modelling,Cleaning,Analytics etc). Also there should be considerations for the associated Risk & Issue Assessment and Management. Get involved in Pre-Sales activities
.Some good resources to write proposals for responses to RFPs seems to be Handbook for Writing Business Proposals
and Persuasive Business Proposals
. Also, the Communication Skills Course
from IIT Bombay can be helpful here.
8) Architectural Discovery and Assessment
– That’s another area that needs to be concentrated.Need to have a 360 degree view of existing Architecture followed by remediation(if any) to minimize the Technical debts.A good start related to all these can be found here
. Knowledge of Assessment related Planning and Governance is also a good to have.
10) Infrastructure Architecture
– Using some thrid party Capacity Planning
Framework or developing some in-house Capacity Planning
Framework(alongwith Hardware Sizing – mainly to decide the disc space and RAM size required) for different Virtualization
options can lead you to an edge.Basic NFRs(Non Functional Requirements) to decide on Capacity Planning(alongwith Hardware Sizing) for a Web Application can be something like Number Of Concurrent Users who will access the application and the SLA(Service Level Agreement) for the processing time to process a page request.
11) Software Architecture – Most of everything mentioned above mainly applies to a Application Architect/Developer but in general, as an Architect (more specifically, a Systems Architect), you are supposed to define the System Architecture as well, where it’s a prime importance that you pursue the qualities to plan and execute NFRs (Non Functional Requirements). Some good resources in this space are Software Quality Attributes, Functional or non-functional requirement?, Understanding Non Functional Requirements, Non-Functional Requirements (NFR) in Agile Practices, Non Functional Requirements – Minimal Checklist, Non Functional Requirements in Software Engineering, Four Layered Approach to Non Functional Requirements Analysis , NFRs – Practices and Recommendations, Quantifying Non-Functional Requirements: A Process Oriented Approach and Architecting High Performing, Scalable and Available Enterprise Web Applications.In-fact, OCMJEA is indicative about the same as indicated here(Phase 3 section). A .NET Developer/Architect can map the OCMJEA specific technologies to .NET(a simple google search can help here) and learn some cool stuffs accordingly.Also as a .NET Systems Architect, expectation w.r.t. planning and execution of NFRs can be for a non-Windows environment as well since lots of viable options have come up as indicated here and here.
Some good resources w.r.t. Software Architecture(& more specifically, Systems Architecture) seems to be Just Enough Software Architecture, Software Architecture for Developers, Software Architecture in Practice, Software Systems Architecture, Documenting Software Architecture, Evaluating Software Architectures (including Tools and Methods for Evaluating the Architecture), DevOps: A Software Architect’s Perspective, Architecting the Cloud, OReilly Software Architecture Series, 97 Things Every Software Architect Should Know, ThoughtWorks Anthology(Vol 2) and Beyond Software Architecture.
An overall nice read is Enterprise Architect vs Solution Architect
A side note on Architecture ->
Architecture involves studying the requirements, choosing architectural style, patterns e.g. layer, MVC, broker, Microkernel. Essentially its about the organization of subsystems in a system and interaction among themselves. Apart from business requirements, quality attributes like performance,availability, scalability, reusabilty, modfiability, testability etc need to be considered as they influence the architecture directly. The inputs for architecture are business requirement, use cases/ stories while output are contextual view, implementation view, process view along-with bench-marking for quality attributes and ATAM (Architecture Tradeoff Analysis Method). Are you a Software Architect can throw some more light in this space.
13) Soft Skills –
a) Ask questions – very important for a consulting job like that of an Architect – Ask Functional as well as System level questions.You might have very good technical in-depth knowledge but if you solve the problems only after you come to know about them, then that is not of much use.Use your technical in-depth knowledge to ask proactive questions relevant to the context.Try to analyze and think about the existing pain points and problems and ask questions accordingly.
b) Enabling a Team ASAP and applying “Divide and Conquer” Principle wherever possible.
c) Be Proactive in everything rather than being Reactive.
d) Try to have a structured(sequential) thought process to solve any kind of problem.
e) It’s important that you have as much knowledge of Technology(Java, .NET, AI, Data Science, Big Data, Software Architecture etc) or the Digital Transformation Movement but probably it’s most important to think about your Client’s Clients i.e. Customers. Always try to make the Customer Experience
as better as you can, with whatever innovative ideas you have(applying Digital Technology maybe).I lag this arena by miles though 😦 . Some case studies showing how firms enhanced their Customers’ Experience via Digital Transformation can be found here
f) And most important – “Think before you act”.
a) Architectural Patterns and Styles
(including Hexagonal Architecture
, Onion Architecture
and Microkernel Architecture
), The Architecture of Open Source Applications
, Application Architecture
, Enterprise Architecture
, Application Integration
, Expositional Architectures
, AWS WhitePapers
, AWS Tips I wish I had known Before I started
, Latency Comparison
, CMS Trap
, Service Design Patterns
, O’Reilly’s Beautiful X Series
(including Beautiful Visualization
and Beautiful Security
), Pragmatic 7 in 7 Series
, Code Project Professional Research Library
and The Pragmatic Programmer
are some awesome resources in general. Architectural talks from .NET Rocks
is also worth hearing. Aforementioned resources are awesome for Application Architecture
and Systems Designing
Microsoft Industry Reference Architectures – Banking
, Capital Markets
, Oil and Gas
, Discrete Manufacturing
, Media and Cable
, Smart Energy
, IoT(Internet Of Things)
, Desktop Hosting
, Cloud Services
, Private Cloud
.Also worth reading are Microsoft Digital Transformation
, Microsoft Digital Marketing
, Microsoft Digital Marketing Platform
, Microsoft Digital Asset Management
, Microsoft SEO
and Microsoft Advertising
g) Whatever mentioned in this article are views based on going through some really good resources and interactions with some beautiful minds at my workplace.If someone has some other view(s)/suggestion(s) then please leave a comment :). And please pardon me for some formatting issues above. There seems to be some problem with the WordPress Editor which I am not able to resolve 😦 .