An AI-first user experience allows users to map and transform thousands of fields from hundreds of data sets in seconds
Brief
Mission
Create a new Duco product to support the company mission to make managing data easy. A product gap was identified early on in the customer data pipeline, we need to fill that gap by taking a wider look at customer use cases. Duco is a disruptor in data management for enterprise financial services companies and the leading AI powered, SaaS platform on the market. The objectives were:
- Increase sales and reduce churn by filling a product gap
- Improve user experience by adding new capabilities to reduce workload
- Support long-term product roadmap by introducing new AI features
Role
Data prep took 12 months to design and develop and I lead the user experience as sole product designer. I advocated for the end users, aligning their needs with the business strategy.
Result
Data prep continues to be a huge success – financially, in user experience and in reputation. Duco now offers a complete operational platform that covers a larger portion of the data lifecycle. Customers love having as many controls as possible in the same platform for cost efficiency, less reliance on IT teams and happier end users. Data prep achieved:
- Standardise multiple data sources and formats with ease
- Flag data errors or omissions at source in real time
- Quickly construct new data sets for seamless transfer
Research
Discovering the customer needs
Large financial companies are bound by law to strict regulatory requirements for activities such as trade reporting. This is currently a complicated and expensive process, involving many different source systems, file formats, teams and platforms.
Customers want a single platform that makes the whole process easier, cheaper and more reliable. Duco already offers the leading reconciliation product, however, it lacks the tools needed for earlier stages of the workflow. Data prep will fill this gap. It will normalise, enrich and transform data into a desired output with automatic quality checks.
Data prep fills a vital gap in the customer journey and brings the entire data pipeline into a single platform
User requirements
Duco is a use case agnostic platform – so data prep must be equally valuable to everyone. From talking to customers, I created broad personas to provide realistic, generalised depictions of users. I discovered who the users are, divided by role and what each of them want from data prep. I used this information to create user journey maps, with the aims of unifying and simplifying existing processes as much as possible.
Data prep fills a vital gap in the customer journey and brings the entire data pipeline into a single platform
Design
Experimentation
Simplicity was my guiding design principle. The challenge is creating a simple user experience while keeping the vast set of features users need. I began by thinking wide, experimenting with a large amount of ideas until I could narrow it down to the best options. Data prep requires a flexible configuration step with many options before users can start seeing results. Wireframes are a quick way of playing with layout to start visualising the end product and simplify the design.
Creating a new data prep process
- Data inputs — The data inputs screen allows users to upload sample data which can can then be mapped to an output schema. The output can be customised to user needs.
- Map and transform — Transform the data using natural rule language or enrich the data with reference data tables. This makes it easy to ensure all the incoming data is normalised.
When creating a new data prep process, users must upload sample input files, any format is supported
The majority of configuration time is spent mapping fields from multiple inputs into a single output, transfornation and enrichment is optional
Advanced configuration
- Triggers — As data arrives into a data prep process it is stored until a snapshot is created to push that batch of data downstream, often into a reconciliation process. Triggers automate the creation of a snapshot.
- Aggregation — Combined data from several rows into a single item based on conditions. For example, sum up all the amount of all trade activity for the same stock in a single day.
The majority of configuration time is spent mapping fields from multiple inputs into a single output, transfornation and enrichment is optional
Results
Lorem ipsum dolor sit amet
The majority of configuration time is spent mapping fields from multiple inputs into a single output, transfornation and enrichment is optional
Validation
To get feedback on the designs, I looked both internally and externally. The Duco professional services team build processes for enterprise customers with the most complex requirements and use cases and will be using data prep every day. So I performed interactive sessions with them and I also distributed a company wide survey.
Externally, I ran interactive sessions with multiple target customers. After 5 usability tests, I had a huge amount of critical insight. Using thematic analysis, I was able to create a list of the most important changes.
Automated set up with AI
After several rounds of design, I decided on a direction and begun working in high fidelity. Data prep consists of 2 parts – creating the process and monitoring the process once live. When configuring the process, I identified that a huge amount of manual work could be removed if AI was used to automatically map fields. For example, a user might want to combine 10 data sets, each with 100 fields. Auto-map will map all fields in just 1 click, all the user needs to do is review the mappings, saving hours or days.
Data submissions arrive into Duco and get routed to the data prep process constantly throughout the day. As data arrives, it is processed according to the configuration and visible in the results screen. Users can review the processed data, checking for volume, format and errors and fix issues before the next snapshot.
Review
Why it works
After 12 months from the idea that sparked the project, the first iteration of data prep was live and being used by customers. Transform data in one platform without code. Duco’s data prep capability can consolidate, normalise and transform data, streamlining processes without the need for code. Flag errors or omissions at source in real time - perfect for highly controlled & regulated financial services environments.
Looking ahead
Lorem ipsum
More projects
Duco
Agentic Workspace
Explore
