PT Calls — Speech-To-Text Pipeline
Automated speech-to-text pipeline that transcribes, analyses, and enriches every call across Bronze, Silver & Gold data layers
In Action
n8n Workflow
Agent Analytics
Call Explorer
Pipeline Overview
Bronze Layer
Data Acquisition
2 steps
Silver Layer
AI Analysis
8 steps
Gold Layer
Distribution
5 outputs
n8n · every 15 min · 7 am – 9 pm
Bronze Layer — Data Acquisition
1
Transcriber
gpt-4o-mini-transcribe
Fetches call recordings from 4COM API, transcribes audio via OpenAI, stores transcription + metadata to S3. Handles rate limiting with exponential backoff and supports date range batch processing.
2
HubSpot Enrichment
hubspot-api
Normalizes caller phone numbers, matches to HubSpot contacts and deals. Extracts client names, deal stages, order numbers, and determines lead group from payment status and lifecycle stage.
s3://bronze/4com/{date}/
Silver Layer — AI Analysis
1
Splitter
gpt-4.1
Segments raw transcript into structured conversation messages with role attribution (customer, agent, bot). Cleans filler words and identifies speaker changes.
2
Analyzer
gpt-4.1
Extracts structured call metadata: summary, next steps, priority score (0–100), follow-up type, payment link detection, and caller relationship identification.
3
Extractor
gpt-4.1
Pulls entities from conversation: client names, order references, postcodes, monetary values. Merges with existing HubSpot enrichment data (HubSpot takes priority).
4
Classifier
gpt-5-mini
Categorizes calls into 12 main categories and 80+ subcategories. Extracts 3–7 key topics per call. Special attention to third-party delivery issues.
5
Lead Scoring
gpt-5-mini
Assigns lead type from 8 types (Pre-Sale, Active Sales, etc.) and buying stage from 60+ stages. Includes sales temperature (Hot/Warm/Cold) and price sensitivity scoring.
6
Delivery Detector
gpt-5-mini
Detects delivery-related conversations and extracts delivery context for downstream processing.
7
Agent Performance & Sales Effort
gpt-5-mini
Scores call handling quality, professionalism, objection handling, and closing technique. Calculates confidence score, stall index, and overall sales effort score (0–100).
8
Validation Protocol
gpt-5-mini
Security and compliance analysis. Checks whether agents verified caller identity, asked permission for non-owner callers (family, installers), and followed data protection protocols.
s3://silver/4com/{date}/
Gold Layer — Data Distribution
AWS Athena
Apache Parquet
Flattens silver JSON to columnar Parquet format, queryable via AWS Athena for analytics.
Pinecone
text-embedding-3-small
Indexes into Pinecone vector DB for semantic call search by the agent platform.
HubSpot Sync
hubspot-api
Pushes call analysis notes, lead types, categories, and sales scores back to HubSpot deals and contacts.
Monday Tasks
monday-api
Automatically creates actionable tasks for agents based on call outcomes — follow-ups, callbacks, payment link tracking, and priority items across Monday.com boards.
Neo4j Ingest
neo4j-driver
Graph database ingestion that builds persistent customer memory. Maps calls, contacts, and deals as nodes with relationship edges for long-term context across interactions.
s3://gold/4com/{date}/
External Services & Infrastructure
4COM API
telephony
OpenAI
AI models
AWS S3
data lake
AWS Athena
SQL queries
PostgreSQL
state management
HubSpot
CRM
Pinecone
vector DB
Monday.com
project mgmt
Neo4j
graph DB
n8n
orchestration
State Management & Orchestration
PostgreSQL Metadata
Idempotent
Tracks every call through each pipeline stage. Each step only picks up pending records, so re-runs safely skip what's already done — fully idempotent with automatic retry on failures.
timber_agents
postgres:5432
n8n Instance
self-hosted
Cron-triggered workflow runs every 15 minutes between 7 am and 9 pm. Steps execute sequentially (Bronze → Silver → Gold), but each step processes calls in parallel using async workers (3–5 concurrent).
*/15 7-21 * * *
async workers
Deployment & Infrastructure
AWS EC2
Docker
GitHub Actions
Runs alongside the agent platform and self-hosted n8n instance on the same machine. Python process invoked by n8n with per-step CLI arguments. CI/CD via GitHub Actions with auto-deploy on push.
t3.xlarge
16 GB
4 vCPU
S3 Data Lake
Three-tier storage following medallion architecture. Bronze stores raw transcriptions + metadata. Silver stores enriched analysis JSON. Gold stores compressed Parquet files.
bronze/
silver/
gold/
Pipeline Summary
3
data layers
16
pipeline steps
8
AI models
5
destinations
~1k
calls / day