Improving the State of Software Development

The fact that the pandemic has accelerated digital transformation for enterprises is no recent phenomenon. However, as the UK emerges from lockdown and releases itself from all restrictions, developer teams face huge challenges ahead when it comes to ensuring organisations are software-ready for a ‘post-pandemic’ world of work. Strategies and overall business plans to keep software development agile must be adapted, but what is the best way to do so? 

Joining us in this week’s QnA to share their thoughts and advice is Dan Fitzgerald, Senior Solutions Consultant at New Relic. Dan has worked at New Relic for just shy of two years, but his knowledge and experience in software development spans back to 2010 when he attended Oxford Brookes University for a BSc in Mobile Computing. Prior to his current position, Dan was employed at IBM as a CICS Tester and Developer before quickly moving into Cloud Technical Sales. 

It’s great to have you here Dan! Your role as a Senior Solutions Consultant greatly resonates with EM360 as we educate the global IT community about the latest enterprise tech solutions. Can you tell us about the responsibilities you have and the types of opportunities/challenges the job has afforded you so far?

In a nutshell, I help enterprise developer teams address their business’s biggest priorities by implementing strong and reliable software development practice. Increasingly, business-critical performance is tied to software outcomes. Just think about the consequences of a major outage – this incident doesn’t just impact the developer team, but the user experience, a business’s reputation, and ultimately, its bottom line.  
  
One of the most common challenges I see when working with customers is a lack of transparency between the developers and the rest of the organisation. With a lot of developers still operating as siloed teams, it is really difficult for them to get a 360-degree view of their software performance. It’s also almost impossible for the rest of the business to get a full understanding of the state of their software and how this is impacting the organisation as a whole.  
  
It’s my responsibility to empower developer teams within their organisations to ensure they’re transparent, data-led and avoid wasted effort, and in turn, get the internal recognition they deserve.  

What is the current state of software development and, more specifically, the software developer role? 

Good software development is no longer a competitive business advantage, but part of a company’s corporate digital responsibility. Today, organisations are completely reliant upon their software, systems, and applications to run a successful business. And, as a result, they are totally dependent on their developers to build and maintain their infrastructure. The developer role has, therefore, become one of the most critical jobs within an organisation but many are yet to get the recognition they deserve. This has to change! Businesses need to double down on their commitment to software development best practice and empower developers in their roles if they are to attract and retain top developer talent.  

From your experience, how has COVID-19 negatively/positively impacted 2021’s initially predicted software development trends and, in turn, today’s developers and the operation of their teams?

It differs greatly, depending on the organisation, the industry, and how badly they’ve been impacted by the pandemic. For many, COVID-19 caused businesses to go into ‘fight or flight’ mode. Developers were consumed with keeping the lights on and the systems up and running. In these instances, whilst businesses might be starting to talk about observability, they are far from implementing it.  
 
However, 18 months on from the start of the pandemic and I definitely see a renewed focus on digital systems across industries. Of course, some organisations are committing to this focus more than others. But as they continue to build resilience against the ongoing turbulence of COVID-19, many are rightly turning to their developers for a successful business recovery. Those leading the charge, are recognising the power of observability when it comes to innovation and having a complete view of a business’ infrastructure. In fact, I’m seeing many businesses now in a surge of hiring designers, project managers and developer specialists in a bid to emerge from the pandemic with not just resilient software, but innovative and competitive online offerings.  

Over the past few years, there’s been a lot of talk about the game-changing use and role of machine learning and AI to make software development agile, ahead of the curve and more impactful. Where are we at with this and what does the future hold? 

AI and ML is becoming more and more common in its use in Enterprise software. Allied Market Research estimates the market for cognitive computing (read that as AI/ML) is projected to reach $87.39 billion by 2026. What this means is that the millions of software developers who work on enterprise software are having to learn new concepts, tools and techniques to implement this and put machine learning models into production. The consequence of this is also an explosion in software companies that help with this problem (known as MLOps). Examples being startups like Seldon, Algorithmia as well as offerings from the big cloud providers like AWS and Google.
 
Another very recent development is the launch of GitHub Copilot. Github is the most popular source code management tool and is the de facto place where open-source code is managed and developers collaborate. They have taken this huge corpus of data and used it to train ML models that can predict the code that a developer wants to write, saving them time and reducing the amount of boilerplate code they need to write. There has been much comment and sometimes criticism online about how this can change the role of the software developer. Is this the end for the software development profession? Of course not, but it certainly has spurred lots of discussion about how it could change the nature of the job, away from churning out lines of code to working at higher levels of abstraction and spending less time writing, more time reviewing code.
 
New Relic has been using AI and ML for years to help our customers make sense of mountains of telemetry data. We’re building systems that emit terabytes of telemetry (metrics, events, logs and traces) per day. No team or individual can hope to grok all of it, that’s where using anomaly detection, pattern matching and other algorithms can aid developers to focus only on real issues and be more productive. New Relic has made amazing progress on this task, from correlating alert data to reduce noise, highlighting patterns in logs to identifying anomalies on any kind of telemetry data. I’m sure there is still more to come in this space.

One of New Relic’s most renowned software-bettering solutions is full-stack observability, which is all about empowering developers with the knowledge and understanding of complex systems. What are the long and short term benefits of embracing this technique for businesses?

Implementing observability enables organisations to get a real-time view of their digital infrastructure. In turn, businesses can make data-driven decisions, adapt quickly to market shifts, troubleshoot problems and provide a platform for developers to innovate, fail, learn and grow.  
  
In the long-term, observability can help empower developers to build a competitive edge for their organisation. By using data-driven insights at every stage of the tech cycle, organisations can accelerate digital initiatives and velocity, whilst reducing toil and operational costs. Over time, this enables developers to build innovation from the ground up.  
  
The short-term benefits of observability can be recognised very quickly. Developer teams that embrace observability fully are able to troubleshoot problems, solve outages and optimise customer experiences very effectively. Therefore, as soon as businesses do run into issues with their software, developers will be able to mitigate and address these problems without impacting the customer experience.  

How can organisations improve their strategies to make software development agile and to ensure they are software-ready for a ‘post-pandemic’ world of work?

The answer lies in embracing observability! However, for software development strategies to be successful, developer teams must have the full business on board. All-too-often I see an assumed tension between the leadership of the business and the demands of the developer community, but this really needn’t be the case. With observability, comes transparency. And to put it simply, it helps to level the playing field between the developer and the wider business. Making the strategy of the business tangible, and the contribution of the developer visible – a win for all. Therefore, the businesses that will pave the way in a post-pandemic world of work will be the ones that don’t just talk about observability but embed these practices across the organisation.

Liked this article? Subscribe to our weekly newsletter for more educational content in enterprise technology.