Data Readiness Assessment

What It Is

Data exists but is scattered, inconsistent, unstructured, and lives in silos. No systematic collection or storage. Information is tribal knowledge or ad-hoc spreadsheets.

Key Characteristics

• No central repository: Data scattered across emails, desktops, departments
• Manual processes: Reports compiled by hand each time
• Tribal knowledge: Critical information exists only in employees' heads
• No data governance: No standards, no ownership, no processes
• High duplication: Same data exists in multiple inconsistent versions

Warning Signs You're Here

• "Let me ask Sarah, she has that spreadsheet"
• New employees can't find historical data
• Different departments have conflicting numbers for the same metric
• Most decisions are based on gut feel, not data

What AI Tiers This Supports

• ❌ No AI tiers reliably supported - Data quality and availability issues will cause any AI project to fail

What You Need to Do First

• Implement basic data storage (shared drives, simple databases)
• Define critical data elements and where they live
• Establish basic data entry standards
• Document key business processes

What It Is

Data is stored in centralized systems (databases, CRM, ERP) but lacks consistent quality, validation, and integration. Basic reporting is possible but requires manual intervention.

Key Characteristics

• Centralized storage: Data lives in databases or business systems
• Basic structure: Tables, fields, relationships exist
• Inconsistent quality: Duplicates, missing values, formatting issues common
• Limited integration: Systems don't talk to each other easily
• Manual reporting: SQL queries or basic BI tools with cleanup needed

Common Technologies

• Basic CRM (Salesforce, HubSpot)
• ERP systems (SAP, Oracle, Microsoft Dynamics)
• Departmental databases (SQL Server, MySQL)
• Basic BI tools (Tableau, Power BI for simple dashboards)
• Excel still heavily used for analysis

What AI Tiers This Supports

• ✅ Tier 0 AI: Automation and rules-based systems (with manual data prep)
• ⚠️ Tier 1 AI: Basic statistical analysis and forecasting (but expect data cleanup overhead)
• ❌ Tier 2+ AI: Machine learning will struggle with data quality issues

Typical Problems

• 20-40% of records have missing or incorrect data
• Same customer exists multiple times with different spellings
• Historical data is incomplete or lost
• Takes days/weeks to compile cross-departmental reports
• Data definitions vary by department

What You Need to Level Up

• Data quality processes (validation, deduplication)
• Master data management (MDM) for key entities
• ETL pipelines to integrate systems
• Data governance framework (ownership, standards)
• Regular data quality audits

What It Is

Data is well-structured, validated, and integrated across major systems. Data quality processes are in place. Automated reporting and analytics are reliable. This is the "AI-ready baseline."

Key Characteristics

• High data quality: Validation rules, automated checks, regular audits
• System integration: ETL/ELT pipelines connect major systems
• Master data management: Single source of truth for customers, products, etc.
• Data governance: Clear ownership, standards, documentation
• Reliable reporting: Automated dashboards, minimal manual intervention

Common Technologies

• Data warehouse (Snowflake, Redshift, BigQuery)
• ETL/ELT tools (Fivetran, Airbyte, dbt)
• Modern BI platforms (Looker, Tableau, Power BI with proper data modeling)
• MDM tools for master data
• Data quality monitoring tools

What AI Tiers This Supports

• ✅ Tier 0-1 AI: Automation and statistical AI (easy implementation)
• ✅ Tier 2 AI: Machine learning (supervised/unsupervised learning works well)
• ⚠️ Tier 3 AI: Deep learning (possible but may need more data volume/variety)
• ❌ Tier 4-5 AI: May need additional unstructured data capabilities

Best Used For

• Predictive analytics and forecasting
• Customer segmentation and clustering
• Fraud detection and anomaly detection
• Recommendation systems
• Churn prediction
• Sales forecasting

Limitations

• Primarily structured data (tables, rows, columns)
• Limited unstructured data handling (images, documents, audio)
• May lack real-time processing capabilities
• Historical data may be limited in scope

What It Is

Comprehensive data platform that handles structured and unstructured data, supports real-time processing, and has mature governance. Can support advanced AI use cases including computer vision and NLP.

Key Characteristics

• Multi-format support: Structured, semi-structured, and unstructured data
• Real-time capabilities: Stream processing for time-sensitive use cases
• Data lake architecture: Store raw data at scale, process as needed
• Advanced governance: Data catalog, lineage tracking, access controls
• Self-service analytics: Business users can explore data safely

Common Technologies

• Data lake (S3 + processing, Azure Data Lake, Databricks)
• Stream processing (Kafka, Kinesis, Pub/Sub)
• Data catalog (Alation, Collibra, Atlan)
• Object storage for unstructured data
• Advanced orchestration (Airflow, Prefect)
• Feature stores for ML (Feast, Tecton)

What AI Tiers This Supports

• ✅ Tier 0-2 AI: All use cases run smoothly
• ✅ Tier 3 AI: Deep learning with images, audio, video, sensor data
• ✅ Tier 4 AI: Most generative AI use cases (RAG systems, document analysis)
• ⚠️ Tier 5 AI: Agentic systems (may need additional tooling)

Best Used For

• Computer vision (quality inspection, facial recognition)
• Natural language processing (document analysis, sentiment)
• IoT and sensor data analytics
• Real-time fraud detection
• Personalization engines at scale
• Video/audio analysis

Who Needs This

• Large enterprises with diverse data sources
• Organizations pursuing computer vision or NLP
• Companies with IoT or sensor data at scale
• Businesses requiring real-time AI capabilities

What It Is

Specialized data infrastructure optimized for AI/ML workloads. Includes feature stores, automated labeling, model training pipelines, experiment tracking, and production ML monitoring. This represents data excellence.

Key Characteristics

• Feature engineering: Automated feature stores with versioning
• Labeled datasets: Systematic labeling workflows for supervised learning
• ML pipelines: End-to-end automation from data to deployed models
• Experiment tracking: MLflow, Weights & Biases for model versioning
• Data versioning: Track data changes that affect model performance
• Observability: Monitor data drift, model decay, prediction quality

Common Technologies

• Feature stores (Feast, Tecton, Hopsworks)
• ML platforms (Databricks ML, SageMaker, Vertex AI)
• Experiment tracking (MLflow, Weights & Biases, Neptune)
• Data labeling (Labelbox, Scale AI, internal tools)
• Model monitoring (Arize, Fiddler, WhyLabs)
• Vector databases (Pinecone, Weaviate, Chroma) for embeddings

What AI Tiers This Supports

• ✅ All AI Tiers (0-5): Everything works seamlessly
• ✅ Production ML at scale: Hundreds of models in production
• ✅ Agentic AI: RAG systems with vector search, tool-using agents
• ✅ Custom LLM fine-tuning: Large-scale training data management

Best Used For

• Companies where AI is a core competency
• Product companies building AI features
• Multiple concurrent AI projects
• Real-time AI-driven products
• Sophisticated recommendation engines
• Autonomous systems

Who Needs This

• Tech companies (AI is the product)
• Large enterprises with 10+ AI initiatives
• Organizations building competitive advantage through AI
• Companies with dedicated ML/AI teams (20+ people)

Your data readiness level isn't a weakness—it's valuable context that helps vendors serve you better. Share your assessment openly in discovery meetings.

What Quality Vendors Will Do

• ✅ Appreciate the clarity and use it for accurate scoping
• ✅ Propose realistic timelines that account for data preparation
• ✅ Suggest whether to address data gaps first or as part of the project
• ✅ Size their team and budget appropriately
• ✅ Set expectations for what's achievable in phase 1 vs. future phases