DA

Senior Data Engineer

Douglas Adam

I build reliable, cost-efficient data platforms — and bring AI into the engineering workflow.

Paisley, Scotland

LinkedInDownload CV

About

Senior Data Engineer with a strong background in data modelling, BI and cloud technologies. I design robust pipelines, optimise SQL and warehouse performance, and build dashboards that turn data into decisions — within clear data quality and governance guidelines, and increasingly with AI woven into the workflow.

What I Work With

Data Engineering & Warehousing

  • dbt
  • Snowflake
  • SnapLogic
  • SQL Server
  • SSIS / SSAS
  • ETL / ELT
  • Data quality & governance

AI Integration & Engineering

  • Python + LLM APIs
  • Agentic workflows
  • Claude Code
  • MCP servers

Programming & Scripting

  • SQL
  • Python
  • REST APIs
  • Automation

Cloud & Big Data

  • AWS
  • Lambda
  • DMS
  • S3
  • Airflow
  • Azure Data Factory
  • Synapse

BI & Reporting

  • Sisense
  • Power BI
  • Qlik
  • Tableau
  • Cognos
  • BusinessObjects

DevOps & Agile

  • GitHub Actions
  • Bitbucket
  • Jira
  • Confluence
  • Azure DevOps

Experience

Senior Data Engineer · EveryLIFE Technologies (Remote)Apr 2023 – Present
  • Actively work with AI technologies to automate data engineering processes, improving the efficiency of delivery and the correctness of data, including agentic workflows integrating the data warehouse, dbt Cloud, Jira and BI tooling.
  • Introduced DBT (SQL) and GitHub to enhance development best practices.
  • Migrated and enhanced pipelines from AWS Airflow to AWS Lambda (all Python), realising a huge cost saving and reduced ingestion run time.
  • Optimized SQL performance in the cloud data warehouse, making queries 2x faster (100% performance improvement) via indexing and schema redesign.
  • Developed solutions enabling near real-time data warehouse updates via streaming and tasks.
  • Strong focus on maintaining the security of sensitive healthcare data.
  • Work closely with the applications team to align data across system applications and the data warehouse.
  • Ensured 99.9% system uptime while integrating data solutions with product teams.
  • Monitored and fine-tuned reporting tools for maximized efficiency.
  • Fully immersed in the AWS cloud tech stack and always willing to learn and use new tools where they fit the use case.
  • Work in agile sprints using the tooling best suited and achieving CI/CD (Jira / GitHub Actions).
Data Engineer · Ocean Technologies Group (Remote)Dec 2021 – Apr 2023
  • Led the migration of report solutions from Microsoft Azure to AWS cloud.
  • Developed integrations using SnapLogic, reducing data silos and improving data accuracy & cost efficiency.
  • Managed operations of the data lake and implemented a new, more robust data warehouse.
  • Worked in a fully Agile & DevOps environment, ensuring continuous delivery.
Business Intelligence Developer · City Facilities Management UK (Glasgow)May 2020 – Dec 2021
  • Integrated Azure Data Factory to streamline ETL pipelines.
  • Developed a data quality monitoring tool to improve data accuracy.
  • Designed Power BI dashboards & SSRS reports to enhance operational insights.
BI Technical Specialist · NHS Greater Glasgow and Clyde (Paisley)Nov 2019 – May 2020
  • Led BI projects, designing new ETL jobs to integrate external systems.
  • Monitored database & reporting tool performance, ensuring system efficiency.
  • Provided BI operational support & server maintenance.
Software Engineer (BI) · JP Morgan Chase (Glasgow)Jan 2018 – Nov 2019
  • Developed compliance & risk management reports using QlikView, Qlik Sense, and Tableau.
  • Optimized financial data visualizations and improved dashboard performance.
  • Provided mentorship to junior team members on best practices and Agile workflows.
BI Developer · DXC Technology (HPE) (Erskine)Feb 2015 – Dec 2017
  • Delivered customized and advanced reporting solutions using QlikView, BusinessObjects 4.0, and Crystal Reports.
  • Acted as the main point of contact for customer reporting queries across global locations.
  • Maintained and enhanced BI development best practices, coaching developers to ensure high-quality, accurate reporting solutions.
BI Developer · Sure Thing Insurance Services (Holytown)Dec 2013 – Feb 2015
  • Built customized BI solutions for Operations, Finance, and Commerce.
  • Led the development of a bespoke reporting suite under strict timelines.
  • Set up a fully automated sFTP data exchange solution for external partners.
BI Developer · BAE Systems (Glasgow)Apr 2011 – Nov 2013
  • Developed and supported BI solutions using Cognos, primarily for Finance and Manufacturing.
  • Provided BI expertise on company projects and managed reporting expectations.
  • Administered the reporting environment to ensure maximum performance and availability.
UK Reporting & System Administrator · Getronics UK Ltd (Inchinnan)Apr 2005 – Aug 2011
  • Developed finance reporting solutions using Cognos BI.
  • Sole UK administrator for the company's ITSM tool.
  • Part of a Global Toolset Support Team, ensuring consistency across regions.
IT Engineer · BMI Healthcare – Ross Hall Hospital (Glasgow)2001 – Apr 2005

Selected Projects

Near Real-Time CDC Pipeline

Change-data-capture pipeline streaming from Aurora MySQL into Snowflake, giving analytics near real-time access to operational data. Built on AWS DMS, S3 and Snowpipe with dbt downstream.

  • AWS DMS
  • Snowpipe
  • Snowflake
  • dbt
  • Aurora MySQL

Snowflake Cost Optimisation

Reduced Snowflake spend through warehouse right-sizing, clustering strategy, dynamic tables and streams/tasks tuning — without compromising query performance or freshness.

  • Snowflake
  • dbt
  • SQL
  • Cost tuning

Interactive Data Lineage Tool

An interactive tool that traces lineage across the stack — Snowflake through dbt models to Sisense dashboards — making impact analysis and debugging far faster.

  • Snowflake
  • dbt
  • Sisense
  • Python

Education

BSc Software Engineering

University of Paisley

2001 – 2003

HND Computing: Software Engineering

Cardonald College, Glasgow

1996 – 1998