Client story

Fish Export Catch Certificates solution

Department for Environment, Food & Rural Affairs (DEFRA)

Situation (the challenge)

The Marine Management Organisation (MMO), sponsored by the Department for Environment, Food & Rural Affairs (Defra), required a new system to create export catch certificates. These certificates would enable fishermen in the UK to export their catches to any EU country post Brexit. This new system needed to be implemented without any disruption, robust in the face of changing Brexit-related policy and created in advance of the original EU exit date of 31 March 2019.

The successful creation and implementation of this service was vital as it is estimated that UK fishermen export around £2.4bn of fresh and processed fish to EU countries. This meant that any disruption to the process would have had a large-scale negative impact on the economy.

Key requirements included having the system work in real time with minimal downtime to ensure continuity. Additionally the system would need to be designed specifically to meet user needs which included a seamless user experience.

Our actions

To create a ‘catch certificate’ or ‘processing statement’ fishermen would be required to log two key pieces of information:

  • species of fish that they had caught
  • name of their vessel

The information of every fish species that can be caught in the UK is kept and maintained by the Centre for Environment, Fisheries and Aquaculture Science (CEFAS) who hold the master species data. However, not everyone calls fish species by the same name so it was mandatory to keep a record of all the common names that the species may be referred to as.

The vessels list is managed by different teams and is not a static list, meaning vessels could be licensed to different companies which requires tracking.

In order to pull the separate data sets and normalise them in a way that enabled quick search was vital. Any time lag in certificate creation would result in degradation in the value of the fish, making operational efficiency key. In addition, the solution needed to be integrated with common components such as the Defra identity management solution and several government departments with disparate data sources.

This ‘catch certificate’ process was non-existent when the UK was part of the EU which led to a major data quality issue due to UK fishermen never having to produce these certificates before. Exports to non-EU countries were minimal so manual creation was manageable and easy up to this point. However the demand for these certificates had drastically increased from approximately 500 certificates manually to 500,000 per year.

The data collected from various departments for species and vessels would be pushed into Azure blob storage and we used our Java backend to collate all of the data into Azure cosmos database. The API exposed by this backend was consumed by the front end in various parts of the journey.

Once all the relevant data to create the catch certificate had been gathered it was pushed into Azure cosmos database. Additionally, we gave the system the ability to generate a PDF with a QR code so security officers at the border are able to quickly scan and confirm the authenticity of the document.

Our idea was to create a Node.js front end. We used React and server side React to support both JS and no JS browsers. The front end would query the back end for all the data and the backend was implemented in Java 8, Spring Boot Service.

The Spring boot application was required to:

  • Look up data in Azure blob storage for species and vessels
  • Ensure species data was updated every few days and the vessels data was updated every night
  • As data was loaded, any issues in the data pulled was pushed into Azure application insights so that the OPS could set triggers to check if there were any issues when loading the data
  • Load all data into the cosmos database
  • On any data lookup, the system should cache data in Redis using Azure Redis Cache
  • If the cache look-up failed only then service went to database. This resulted in a big boost to performance as the majority of fishermen exported the same fish to different countries. The ability to search on the common names of the species, FAO code or scientific name was massively useful for the end-users.
  • Ensure all documents generated in the system also went into the cosmos database. This gives the user the chance to go back and look at all the historic documents that user had generated previously.

Outcomes

The solution devised and built by the Scrumconnect team was available well in advance of the original deadline of 31 March 2019. AllBbeta users found the system to be intuitive and quick to generate certificates. We integrated the external systems with the Azure app which gave us insights and detailed monitoring which meant that if there were any challenges with data, the issue could be identified and rectified quickly. As we designed this solution from the ground up, we ensured it would be ultra efficient when querying for data. This meant that the solution was able to go well beyond the expected number of requests from the system in our stress tests.