General Principles - Software Company
Creating high-quality software requires robust processes and procedures to maintain and monitor standards.
These principles guide the development and delivery of high-quality software to customers and create the software development lifecycle (SDLC).
In addition to software development, the company is required to maintain broader principles and IT policies that govern acceptable use, confidentiality, and employee onboarding (including recruitment) and offboarding processes.
Mandatory (enforced) policies and procedures are highlighted where applicable.
The playbook/template is broken down into categories:
- General Principles - Software Company
- Code Quality and Standards
- Code Programming Language
- Coding Standards/Best Practice
- Documentation
- Testing
- Agile Development Practices
- Security First Approach
- Secure coding practices
- Regular Security Reviews/Upgrades
- Technical Excellence
- Scaling
- System performance monitoring and optimization
- Technical Debt Management
- Technical Reviews & Architecture
- Financial Awareness
- DevOps Culture
- Local development
- Infrastructure as Code
- Trunk Based Development
- Don’t forget to version control SQL databases
- Continuous Integration
Code Quality and Standards
High-quality code and consistent standards provide numerous benefits to software development overall improving the readability of the code and making it easier to maintain and understand.
Code Programming Language
Where possible adopt industry standard toolsets and languages which are supported by a large company or have significant open source community, have a large following and actively improved over time.
Typical languages include C#, SQL, Python, Java etc.
Avoid languages which require royalty payments or licences for usage - this can include some versions of Oracle Java.
The above guidance is not applicable in a retrospective manner with an already established product.
Coding Standards/Best Practice
It isn’t possible to provide a single set of pre-defined standards which would be applicable to all due to the variety of technologies in existence. However each company is required to:
- Adopt a coding standard and ensure it is documented. Standards will vary based on your industry and technology stack. Instead of creating new standards, use existing industry-standard ones when possible. Consider adoption of CERT standards. Useful reference
- Coding standards should be introduced to developers during their onboarding and induction process.
- Where possible enforce these standards through:
- regular (human) code reviews
- automatically during code check-ins using static analysis tools like SonarQube - rejecting code that doesn’t meet the standard.
- Standards should be consistently applied across all projects & products (this is easier if using a tool)
Documentation
Comprehensive documentation must be created for all code and systems, this ties in with the requirements in
Documents must include:
- Diagrams of the system components - adopt the C4 model as a standard 4 layer model - https://c4model.com/
- Instructions on how to build and compile the code, including how to setup a developer machine to do this.
- Instructions on how the code is deployed and configured into a test and production environment (for example pipelines or manual configuration)
- Documentation on what infrastructure is needed to support the application
- Documents covering any overly complex or intellectual property (IP) parts of the code/system.
Testing
Ideally testing is carried out by both developers and test engineers, both skillsets compliment each other to provide a wide coverage of scenarios and technical tests.
Testing must cover:
- Unit testing of code modules - typically through an automated test framework at build time. These tests are typically written by developers.
- Automated testing of the system - both via the user interface and through the use of test harnesses for API services.
- Introduction of “mocks” and “stubs” to emulate 3rd party services to aid testing.
Agile Development Practices
Portfolio companies should adopt an agile way of working. Agile is a mindset rather than a predefined template—it involves a recurring cycle of events where teams learn from both successes and failures in each iteration. This approach is founded on the principles of the Agile Manifesto from the early 2000s. https://agilemanifesto.org/
SCRUM is a well understood and common implementation of Agile - further background reading is available here.
Each portfolio company is required to adopt SCRUM for their development process. They will be required to:
- Maintain an up-to-date Product Backlog of prioritized tasks, features, or requirements for the product. In larger companies this should be managed by the product owner.
- Hold daily stand up meetings to ensure the sprint is on track and remove blockers
- Hold retrospective meetings after a sprint to review the work produced and highlight successes and failures - provide continuous feedback to people involved to improve quality.
- The agile process is not just for developers - include multi-discipline team members such as testers and cloud/infrastructure engineers, they all form part of the success of the project.
- In sprint planning meetings, use story points for estimating time and complexity.
- Regularly review the allocation of story points and the actual effort involved to optimise productivity (sprint cadence)
A suitable tool is used to hold the product backlog and organise sprints - JIRA or Azure DevOps Boards are the recommended tools.
Security First Approach
When designing systems, always consider security as a first principle. Think like an attacker: what data might the system or service expose, and how can you prevent unauthorized access?
Secure coding practices
A basic background in secure coding practices is available from here https://mlhale.github.io/nebraska-gencyber-modules/intro_to_first_principles/README/
In particular, consider Domain Separation, Least Privilege and Simplicity as key factors in your software design.
- Never design your own security or encryption mechanism - these should be left to the experts, instead adopt well known and tested libraries such as OpenSSL or Microsoft Crypto.
- Your code must only be given enough privilege to operate (least privilege) - if it doesn’t require access to a database or network folder, don’t grant it permission.
- Keep things simple and avoid complexity, as it becomes increasingly difficult to test complex systems and prove they are secure.
Regular Security Reviews/Upgrades
You must regularly update the software to include the latest security fixes of 3rd party libraries such as the .NET framework and the various NUGET/NPM packages that exist.
These should be regularly tested and deployed even if no other code changes are required to maintain a secure and cared for environment
Testing Tools
Use security testing tools such as AppCheck and SonarQube to verify that your code and its dependencies are free from vulnerabilities and operating as intended.
Technical Excellence
Once a solution has been built and deployed, it needs regular maintenance to avoid becoming legacy and a technical burden to support.
Scaling
How will your system scale? Has it been designed and written to run in a multi-threaded and multi-server environment?
In the frequent usage of cloud hosting, your application must be able to operate in a stateless manner split across availability zones to provide seamless experience to the end user should a zone fail.
System performance monitoring and optimization
Deploy monitoring tools to check performance and resource utilisation of a system. This also aligns to being financially aware and optimising resources - particularly in cloud environments.
Alerting should be configured to provide infrastructure alerts to engineering teams. Regularly review which metrics are important and establish a normal base line value for each.
Uptime monitoring is critical if your service is contracted via an uptime SLA guarantee.
Technical Debt Management
Review lifecycle of 3rd party dependencies (.NET framework etc.) and ensure you are always running on a supported, current version.
Regularly (quarterly at minimum) update the 3rd party dependencies your code relies on, even if there are no other coding changes required at that time. Test the change and deploy it - this aligns with a security first approach.
Technical Reviews & Architecture
Keep abreast of changes in technology which could benefit your solution, either reducing cost or improving performance are good drivers to adopt a new technology.
Avoid “jumping on the bandwagon” of new technical trends, instead evaluate how they could benefit (or not).
If there isn’t a compelling reason to change, then don’t - however don’t get left behind and ensure your 3rd party dependencies are always in support and regularly patched.
Financial Awareness
Developers don’t normally need to consider the financial impact of their decisions, however in a cloud deployment the choice of technology or server specification can have a substantial impact on cost.
Before a technology is chosen, look at the cost of supporting and maintaining that technology. Plan for growth, where will that cost be in 1 or 3 years time, consider the impact of new customers and organic growth over that period.
Regularly right-size resources deployed into cloud providers. Review the billing on at least a quarterly basis and look for upward trends. Review the performance of each cloud service and determine if you have over allocated resources which could be trimmed down.
Adopt the mindset of “just enough” to perform the task - with the cloud its easy to scale up, so no need to over provision.
Review cloud provider technology stacks for cost optimisation through alternative technologies. You may have manually implemented a technology which is provided by a native cloud solution, consider adopting those native services rather than “rolling your own”.
Database technologies are often very expensive, particularly Microsoft SQL Server, where possible determine if you can use an alternative which will be as performant and lower cost to operate.
DevOps Culture
Local development
Developers should work on their own isolated development environment - normally on a local desktop/laptop.
Databases should also be developed locally, as recommended by Microsoft, to aid productivity for developers to avoid conflicting with each other if using a shared database. This also avoids network latency of using a remote database server over the Internet or VPN.
Scripts should be created to allow this environment to be rapidly destroyed and recreated - including sample test data mimicking a production environment.
Infrastructure as Code
Treat the infrastructure as if it were lines of code, use custom tools like Terraform to define what infrastructure you require for the application.
Never make manual changes to the infrastructure, always do this via the tool/code, and version control the scripts.
You should never be afraid of destroying the environment - it should be easy to recreate if the scripts are correct.
Trunk Based Development
Adopt Trunk Based Development in your version control system (GIT) - https://minimumcd.org/minimumcd/tbd/
It works for large organisations like Microsoft and Google, so should work for everyone.
Don’t forget to version control SQL databases
Database schemas are critical components of your codebase and should be version controlled.
Use tools like Redgate SQL Compare and https://dbup.github.io/ to generate incremental upgrade scripts and version control them.
It should be trivial for a developer to update the local development database to reflect changes other developers have made.
Continuous Integration
Adopt CI best practice https://minimumcd.org/minimumcd/ci/