Twenty projects, three decades, one discipline

Automated executive dashboard

Overjet · dental AI · team of four on GCP, Looker, and Sigma

Challenge

Preparing leadership presentations for the company, investors, and shareholders was slow and laborious. The most relevant facts had to be assembled by hand every time, at varying levels of importance and detail.

Approach

Defined a manageable set of priorities that lets the key facts be generated automatically at whatever level of brevity is needed. Beyond saving time, this improved the clarity, completeness, and quality of what leadership saw.

The ideal data function for a SaaS business

SimplePractice · Head of Data · team of 12

Challenge

Most organizations stop well short of harnessing their data because the data function is poorly defined. Data is everywhere in the company, but priorities and ownership are siloed by department.

Approach

Drawing on experience across every aspect of data, carefully delineated what a central data office should own, oversee, and manage: enterprise data architecture, the application database platform, governance, and insights. Led analytics for Product, Marketing, Customer Success, Risk, and the executive team, including real-time fraud and risk reporting with Stripe and Kount.

"Data Function for a Successful SaaS Organization"

Analytics for a multi-cloud, multi-engine big-data platform

Qubole · Senior Director of Analytics · team of three

Challenge

Normalizing analytics for a service that runs on four clouds is hard: cluster management, services offered, and pricing all differ from AWS to Azure to GCP to Oracle Cloud.

Approach

Developed a series of data-modeling innovations to normalize usage data from four clouds, thirteen environments, and hundreds of customers into one company-wide data lake, then delivered analytics and visualizations that every audience could read, from analysts to management to investors.

Monitoring dormant accounts for resurrection

Facebook (Meta) · with one teammate · sharded MySQL and DataStorm (Facebook's Airflow)

Challenge

An online advertising system manages accounts for a vast number of content providers and needs to re-engage the ones that have gone quiet, at the right moment and with the right message.

Approach

For each dormant account, the system checks whether a past campaign performed well against a benchmark such as a percentile of cost-per-objective for its country, page category, and objective type. If so, it generates a recommendation that highlights the earlier success and links directly to purchasing another campaign. Granted as US Patent 2018/0096378.

Read the patent

Consolidation and advanced analytics of state-wide real estate data

MLS Listings, Silicon Valley · team of 12 · SQL Server and Power BI

Challenge

Real estate listings and sales in the United States are recorded by hundreds of independent multiple listing services (about 900 at the time). Two problems: how to consolidate copies from several of them efficiently, unifying structures and timing, and how to deliver advanced analytics on trends, optimal pricing, and other metrics on the result.

Approach

Built a data warehouse and an analytics platform and advised the executive team on business expansion through BI technology.

Database design evaluation for a medical devices application

ResMed · $4.8B revenue medical devices company · SQL Server

Challenge

A flagship application built by a third party was low quality and slow. A comprehensive, independent evaluation of its application and database design was needed, with concrete fixes.

Approach

Devised evaluation criteria across robust modeling, naming, logic placement, integrity, and performance and delivered a full assessment. Proper logic placement and well-written SQL then produced performance improvements of several orders of magnitude. Also modeled information for the company's key products: flow generators, masks, and wireless devices.

Enterprise data migration

Equinix · $8.6B revenue data center company · led a team of 30 across California and India

Challenge

Define the architecture and manage the migration of enterprise functionality across tens of applications, including master data management and real-time integration over a bus architecture, while new software, new versions, data-quality errors, and high transaction rates kept arriving.

Approach

Moved functionality from legacy systems including Oracle EBS and Salesforce to Siebel UCM/CRM, TIBCO middleware, Singleview, Vertex, and Apttus, with heavy focus on ETL. Worked with eight cross-product teams covering MDM, billing, capacity management, operations, install base, reporting, sales, portals, and integration.

Unified data management for disparate event-scheduling systems

Active Network · Director of Data Engineering · team of 20 in the US, Canada, and China

Challenge

A collection of systems accumulated through acquisitions ran in different states on different platforms and technology stacks. They needed performance improvements, capacity planning, and uniform consolidation to share data and give users a seamless real-time experience.

Approach

Defined the critical disciplines of data engineering (archiving, performance, model change management, multi-tenancy, migrations, ETL) and proposed a loose integration approach built on an information bus, where a unified enterprise model drives definitions and mappings across products instead of costly, risky hard integration.

Unifying applications built on different architectures

Concerro, now part of GE Healthcare · Senior Information Architect

Challenge

Two key applications had incompatible database backends, MySQL and Oracle, but needed to become one product, with as little application code change as possible.

Approach

Evaluated several database vendors and features, then evangelized, designed, and implemented a database-centric integration platform that combined the two seamlessly. A complete overhaul of database access methods turned a customer-churning product into a high-performing one, which helped enable the company's acquisition by API Healthcare.

Database and data-access layer for a workforce-forecasting application

Verint · multi-tenant SaaS for banks · SQL Server and Oracle

Challenge

A multi-tenant forecasting application for banks needed a unified data model, database design, and data-access layer that delivered high performance while keeping each customer's data absolutely separate and protected. Built into the system were modified queueing models of bank operations.

Approach

Designed and implemented a high-performing database and a secure multi-tenant data layer in ADO.NET and C#, with a performance roadmap that produced multifold capacity gains within twelve months and was transferred to the company's global units.

Pilot hormonal-health website

Lucine Biotechnology · Chief Technologist · team of 10 · Java, DB2, SQL Server

Challenge

Build a website where women can monitor hormonal changes, communicate with experts, and collaborate on hormonal and other aspects of women's health.

Approach

Assembled and led a ten-person team working for equity, and did the hands-on data modeling, architecture, and database design, integrating a model-driven approach into the development process.

Patient-physician encounter support software

MedAnt · CTO · team of three · MySQL

Challenge

Support internists during patient encounters with an expert system that helps assess symptoms, validate a diagnosis, and propose treatment plans. A central difficulty was modeling human anatomy, a hierarchy of illnesses, medications, and other health data.

Approach

Advised on the platform and developed a large-scale web application for clinical support, working with one developer and one subject-matter expert.

Redesigning an automated voice-mail delivery database

Voice-mail broadcasting company · Senior Manager and Head Database Architect · combined team of 30

Challenge

A system that schedules and places automated telephone calls had an unacceptable delay in daily data preparation. Billions of records had to be reviewed every day against do-not-call lists at the federal, state, and company level.

Approach

Overhauled the database design, optimized field sizes, and applied proper indexing to reach a ten-fold performance improvement on databases of over three billion records, which made the company able to compete for Fortune 500 clients.

Enterprise data warehouse and analytics

HSBC Auto Finance · Head of Information Management · 40+ staff across San Diego, Chicago, Tampa, and Bangalore

Challenge

Reports lacked quality and consistency and took extraordinary effort to deliver daily. Sources were disparate, from a mainframe operational data store to several Linux systems, delivered through Actuate and Cognos.

Approach

Organized the department into Decision Support, Data Support, Data Quality, and Corporate Initiatives groups, streamlined processes, and improved the technology stack with QualityStage. Efficiency doubled to 30 projects a month and 60 concurrent pipelines, supporting finance, risk, marketing, sales, credit, and collections.

Brute-force matching of person records

Illinois Department of Human Services · $4B program · massively parallel DB2

Challenge

A welfare department distributing medical, food-stamp, and cash assistance must be certain that no individual is unknowingly counted twice. Robust de-duplication of clients is essential.

Approach

Built a distributed matching application on record linkage theory, running on a massively parallel database. It improved matching accuracy three-fold and surpassed the leading commercial tool, Vality Integrity (now IBM QualityStage).

"Record Linkage Theory," SDSU colloquium

Automated ingestion of legacy client data

Illinois Department of Human Services · mainframe to DB2

Challenge

The primary data source was a still-operational legacy mainframe database using hierarchical, non-ASCII storage, with data structures that changed over time.

Approach

Developed an automated solution that reads the COBOL copybook data definition and generates an ETL program in C, delivering both speed of implementation and flexibility across structure versions.

Massively parallel data warehouse for a welfare system

Illinois Department of Human Services · Chief Technical Architect · 19-node IBM SP2, DB2 EEE, 2 TB

Challenge

A large population of welfare recipients and high transaction rates for dispersing assistance required a robust, high-performing data warehouse serving over a thousand users.

Approach

Planned, configured, designed, and implemented the entire environment: operational data store, staging, central warehouse, data marts, cubes, and reports, with data partitioned and replicated according to a strict distribution discipline. The system cost about $8M over five years versus $30M for a smaller system at another state agency.

"Due to Farid's expertise, selection of proper technologies, and automation-oriented development, our data warehouse costs only a fraction of the cost of a much simpler, smaller, and less functional data warehouse developed by another state agency. This has saved us millions of dollars."

Offline system for processing airline fares

Sabre · Chief Architect · team of six within a program of 30

Challenge

A fares management system had to be built from scratch, driven by Y2K and the inflexibility of the legacy platform. Hundreds of hierarchical data structures represent airline pricing rules, and the new system had to integrate seamlessly with a limited clone of Sabre built for a client.

Approach

Three parts: normalized the hierarchical rule structures into a relational model; generated user-interface screens automatically from database metadata; and generated in-memory versions of the pricing data that the legacy clone could read. Chosen over two competing Sabre architectures, the automation produced 75% profitability on the roughly $10M subproject, which carried the entire $50M program.

Performance benchmark of a relational reservation system

Sabre · Technology Manager · Digital and Oracle

Challenge

Prove that a proposed relational version of the airline reservation system could match the performance of the legacy mainframe system.

Approach

Developed a benchmark that reached a simulated 95% of the real Sabre load, around 700 transactions per second, on a single powerful server, drawing direct interest from Oracle's Larry Ellison. Earned Sabre's 1995 Circle of Gold award for the top 2% of performers.

Converting a legacy reservation system to relational

Sabre · Lead Architect, Technology Planning and Architecture

Challenge

Demonstrate that reservation data could be modeled relationally and that the new and legacy systems could coexist during migration.

Approach

Built a prototype that compiles the unstructured passenger name record on the fly into SQL inserts on a separate Oracle server, virtually instantly, and converts an application's cryptic command syntax into SQL reads and writes against the same database. Both syntaxes follow a grammar, so Unix tools (lex and yacc) automated the conversion. The work produced two US patents.

US 6,122,642 · US 6,714,945