Progress MarkLogic Makes Logistics Less of a Battle for Marine Corps with Data Modernization

TDM-Catalyst was a major Logistics IT modernization effort for the United States Marine Corps (USMC), which handles loads of military materials and equipment, or materiel, critical to their mission. The catalogs of information needed to manage that materiel— including parts for repair and replacement—are the subject of the TDM-Catalyst project.

The legacy applications involved in this modernization effort included four separate catalog and procurement enterprise information systems. These legacy systems were notoriously slow and unresponsive, had no coordination or exchange of information and impeded accuracy. Leveraging a decades-old mainframe and mid-tier computing infrastructure technology, these systems forced users to perform excessive manual data entry, risking human error and man hours. For example, when adding an item with a National Stock Number (NSN), users had to update 12 different systems—a process that took six to eight months to complete and resulted in frequent errors, out-of-date information and duplicate orders. The process was antiquated and inefficient.

IT modernization was a goal for the Deputy Commandant, Installations and Logistics (DC I&L). They created the Logistics Information Technology (Log-IT) branch of the Logistics Command to improve mission operations using current IT resources like cloud computing and web browser user interfaces.

The goals of TDM-Catalyst are to transform Log-IT:

• Streamline and automate the catalog and provisioning processes

• Establish a cloud-based data platform that has full accreditation for DOD use

• Ingest catalog data from various sources using a modern multi-model/NoSQL data platform that can continuously and flexibly manage change in the data structure

• Improve data quality and clean and master the data for TDM-Catalyst and other downstream user requirements

• Enable future modernization projects with reusable accredited technology

• Foster data reuse and continuous improvement of processes 

The U.S. Marines needed a streamlined system that would give their personnel full visibility into the inventory to provision parts and equipment for the warfighter in a timely manner. 

The preliminary work involved a Technical Data Pilot (TDP) project using Agile development practices and a trial of the Progress^® MarkLogic^® Data Hub and Appian’s low-code development platform. The resulting system was determined to be an ideal environment and the initial pilot was a success. In the TDP, the TDM-Catalyst team found that this modern technology could overcome the limitations of relational databases—namely, the ingestion of disparate data schemas and formats and the creation of usable data from undefined sources.

Eric Bower, Technical Lead, said, “We had to effectively build a Rosetta Stone on the backend.” The ability to speak in different languages, in terms of data, enabled rapid synchronization of business processes and data structures. Bower added, “We needed immense flexibility as we went through development to harmonize all of the data from a wide range of sources and pedigrees... And we didn’t want to get stuck rebuilding schemas because we had no clue what everything was going to look like going into it.” The multi-model MarkLogic Data Hub “made perfect sense.”

The team achieved full accreditation and DOD Authority to Operate (ATO) for the TDM-Catalyst system on AWS. This was the first USMC Program to achieve this milestone and the first USMC cloud-based application. The MarkLogic and Appian software platforms, and the TDMCatalyst application itself, are entirely hosted on AWS and approved. The accreditation can be reused by other DOD entities. Others who wish to use the same technology can do so with their own cloud environment. 

The TDM-Catalyst effort will require adjustment along the way and throughout the program lifecycle. The MarkLogic platform enables Agile development and fast modifications to the data without disrupting the operation. For DevSecOps, the MarkLogic platform is the perfect answer. It tracks the changes to data, maintaining provenance and lineage. 

To enable modernization for future IT projects, the team designed and implemented the system with other use cases in mind. TDM-Catalyst was architected and built within MCBOSS and USMC requirements that can be applied to other data modernization projects using DevSecOps processes.

Agile software development practices are central to the TDM-Catalyst modernization, requiring subject matter experts to act as stakeholders throughout the process and using “sprints” to detect errors and design flaws early so they could be fixed before they got worse.