# Changelog

All notable changes to Anwal Growth Platform are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) principles.

## [Unreleased]

### Added

- **White Label & System Branding (Task 023):**
  - Settings → Branding — platform/company name, logos, colors, website, support contacts
  - `BrandProvider` + `useBrand()` — sidebar, login, dashboard, marketing home read from settings
  - Brand asset upload to `uploads/branding` with public serve route
  - Centralized email footer (`buildEmailFooterHtml`) from brand settings
  - Email template defaults and starters use `{{company_name}}`, `{{sender_name}}`, `{{sender_email}}`
  - AI brand voice and campaign sender fallbacks use branding settings
  - Docs: [28-white-label-branding.md](./28-white-label-branding.md)

- **CRM Timeline Completion (Task 022):**
  - Extended activity types: campaign_*, sequence_*, email_received/archived, note_created, ai_scored, pipeline_stage_changed, lead_converted, lead_score_changed
  - Enhanced `activity.service.ts` — search, category filters, counts, notes API
  - Unified timeline on Company, Person, Lead profiles with summary cards and note form
  - Global activity center at `/activities`
  - Task create/complete and AI scoring log timeline events automatically

- **AI generation language + brand voice integration:**
  - Required Language selector (Arabic / English) on Email Writer, Template Generator, Sequence Generator, Campaign AI Writer
  - Default language from Brand Voice `preferredLanguage` via `GET /api/ai/guard`
  - Arabic output: RTL, Arabic-only copy; English output: LTR, English-only copy
  - All generation uses brand voice settings (no hardcoded Anwal / sales@anwal.sa in AI prompts)
  - Template builder apply instantly updates subject, body, CTA, direction, and live preview

- **AI Content Generation & Lead Intelligence (Task 021 Phase 2):**
  - Brand Voice settings (`/settings/ai`) — company context, tone, CTA, signature, forbidden claims, preferred language
  - AI Email Writer in templates, campaigns, and sequence builder (`POST /api/ai/email-writer`)
  - AI Template Generator with 14 template types (`POST /api/ai/template-generator`)
  - AI Sequence Generator on `/sequences/new` (`POST /api/ai/sequence-generator`)
  - AI Segment Builder on `/segments/new` — natural language → filters (`POST /api/ai/segment-builder`)
  - AI Lead Scoring on Lead, Company, Person — single + bulk (`POST /api/ai/lead-scoring`)
  - AI Next Best Action on CRM profiles and inbox thread panel (`GET /api/ai/next-best-action`)
  - AI Follow-Up reply suggestions in inbox and lead actions (`POST /api/ai/follow-up`)
  - `AiActionLog` audit trail for all generation features
  - `checkAiGuard()` graceful handling when AI disabled or API key missing (rule-based fallback, Arabic messages)
  - Docs: [26-ai-content-generation.md](./26-ai-content-generation.md)

- **Professional Inbox — multi-mailbox support:**
  - `EmailThread.mailboxId`, `mailboxEmail`, `mailboxName`; thread list filtered by mailbox
  - `/inbox` mailbox selector (`كل الصناديق` / default / per-mailbox); URL `?mailboxId=` / `?mailbox=all`
  - `GET /api/inbox/threads` accepts `mailboxId` and `mailbox=all`
  - `POST /api/inbox/sync` accepts `{ mailboxId?, all? }`; defaults to default active mailbox (not `.env` IMAP)
  - `resolveInboxImapOptions()` prefers DB mailboxes; Arabic errors for missing/incomplete mailbox IMAP
  - Backfill assigns `mailboxId` to threads/replies; single-mailbox fallback for legacy data
  - Cron `sync-mailboxes` unchanged: always syncs all active mailboxes

- **Professional Inbox & Conversation Center (Task 020):**
  - `EmailThread` model + extended `EmailReply` (threadId, direction, isRead, CRM links)
  - Thread matching on sync; `GET`/`POST` `/api/inbox/threads/**` APIs
  - Three-column `/inbox` UI: thread list, conversation bubbles, CRM + AI panel
  - Reply from system, archive, classify, stop sequence, suppress, create task
  - `email-body-cleaner.ts` — collapsed quoted history, RTL/LTR
  - Backfill: `POST /api/admin/maintenance/backfill-inbox-threads`

- **Campaign email policy settings:**
  - `SystemSetting.emailPolicy` — `duplicateWindowDays` (0–365) and `duplicateAction` (`warn` | `skip` | `allow`)
  - Admin UI `/settings/email-policy` + `GET`/`PATCH` `/api/settings/email-policy`
  - Campaign send respects policy: `0` disables check; `skip` auto-skips; `allow` sends without modal; `warn` returns 409 + UI decision modal
  - Fixed campaign UI: 409 with `requiresDuplicateDecision` opens modal (not error toast)

- **Sequences & Smart Outreach Engine (Task 019):**
  - New `/sequences` module (separate from Campaigns): multi-step outreach builder
  - Models: `Sequence`, `SequenceStep`, `SequenceRecipient`, `SuppressionList`
  - Execution engine + cron: `POST /api/cron/run-sequences`, `POST /api/cron/sync-mailboxes`
  - Recipient generation via `getAllDedupedSegmentMembers` / batched segment dedup
  - Stop on reply/click/open; AI reply classification; suppression on unsubscribe/bounce
  - Campaign → Sequence action; `segment-dedup-accumulator` wired into preview/members/campaign/sequence paths
  - Processed import aligned with Simple Import cleaning rules

- **Task 019 finalize — operations & UI:**
  - `CronRunLog` model + cron job service; cron endpoints log every run
  - Admin cron APIs + `/settings/cron` (summary, logs, manual mailbox sync / sequence worker)
  - `/sequences` AppShell layout + middleware protection (sidebar/topbar parity with Campaigns)

- **CRM Foundation Hardening (Task 018):**
  - Unified CRM linking service (`crm-linking.service.ts`)
  - Import contact-channel eligibility rules + email classification on contact points
  - `email-validation.ts`, ContactPoint validation fields, campaign `sendOnlyValidEmails`
  - Segment `targetChannel` (email/mobile/whatsapp/any) + channel-aware dedup
  - Global send dedup: `email_send_history`, duplicate window settings, pre-send warning modal
  - Activity timeline service + `GET /api/timeline` + profile/campaign UI
  - Admin maintenance backfill: `POST /api/admin/maintenance/backfill-crm-foundation`

- **Production-ready import engine:**
  - Smart column mapping (simple), deduplication (website/LinkedIn/name/fuzzy), enrichment (fill missing fields only)
  - Import progress stages on `ImportLog` + `GET /api/imports/status/:importLogId`
  - Async execute via `after()` — UI polls progress; prevents double execution
  - Compact import summary and history (created/updated/skipped counts, no huge JSON dumps)
  - Processed import: same dedupe/enrichment; optional sheets only (Entities, People, Contact_Points, Service_Matching)

- **Simple import smart auto-mapping:**
  - `import-mapping.service.ts` — semantic column detection (English/Arabic/mixed), value-based inference, confidence scoring
  - `POST /api/imports/detect` returns `suggestedMapping`, `mappingMatches`, `unmappedHeaders`
  - Preview/execute auto-map when mapping JSON empty; response includes `mappingUsed`
  - `/imports` UI: auto-fill mapping on detect, confidence + matches table, Arabic review note

- **Entity profiles & smart segments (Task 018b):**
  - Company profile (`/market/companies/[id]`) — people, contacts, service matches, campaign history, inbox replies, leads, opportunities, activities; quick actions (segment, lead, task)
  - Person profile (`/market/people/[id]`) — contacts, campaigns, replies, leads, opportunities, activities; quick actions
  - Contact points table — linked company/person, campaign/reply counts, clickable links
  - Segment filter options API — dropdown values from imported data
  - Static segments — `type: static`, `POST`/`DELETE` `/api/segments/:id/members`, member manager on segment detail
  - Bulk “Add to Segment” on market list tables (static segments only)
  - APIs: `GET /api/market/people/:id`, `GET /api/segments/filter-options`, `GET /api/segments/static-list`

### Fixed

- Processed import: map `entity_name_ar` / `entity_name_en` columns to companies; upsert by website/name; fill `entity_id` map for people/contacts; preview email/mobile counts; execute API always returns JSON (`maxDuration` 300s)

### Removed

- Data Quality and Duplicate Review modules from UI and import workflows (`/market/data-quality`, `/market/duplicate-reviews`, related nav, dashboard metrics, market overview cards). MongoDB models/collections retained for legacy data. Cleaning/duplicates expected in GPT preprocessing before import.

### Changed

- **UI/UX redesign (Anwal branding):**
  - Design system: Tailwind tokens, shadcn-style components, Inter + Noto Kufi Arabic, light/dark/system theme, AR/EN locale
  - Dashboard: real data insights via `GET /api/insights` (no fake AI cards)
  - Instant URL-synced filters (no Apply) on market lists, leads, campaigns, segments, inbox, notifications, tasks, templates
  - Tabbed CRM profiles for companies and people; modern lead/opportunity/inbox/campaign/segment detail layouts
  - Shared hooks: `use-instant-list`, `use-filter-params`; login page rebranded

- Email test page: SMTP verify/send via environment or selected mailbox (`/api/email/mailboxes`, transport `source` param)
- Mailbox credentials: fix password persistence (`select: false` reload), `hasSmtpPassword`/`hasImapPassword` flags, trim/encrypt on save
- Mailboxes settings: delete mailbox (UI + existing `DELETE /api/settings/mailboxes/[id]`)
- Mailbox save: form always sends SMTP/IMAP user fields; PATCH uses field presence so partial updates do not clear credentials; `$set` + reload after update; diagnostic flags (`hasSmtpHost`, `hasSmtpUser`, etc.); SMTP transport reads credentials via `getMailboxSmtpCredentials`
- Inbox IMAP verify/sync: environment or selected mailbox (`source` / `mailboxId`)

### Added

- **Task 018 completed — v1.0 internal testing readiness:**
  - Security: permission guards (`canRead`/`canWrite`), viewer read-only, admin routes unchanged
  - API helpers: `api-guards.ts`, enhanced `api-response.ts` (safe production errors, logging)
  - HTML safety: `stripDangerousHtml` for inbox and template preview
  - Health: `GET /api/health` — version, SMTP/IMAP presence, AI status (no secrets)
  - DB: removed duplicate `users.email` index; `scripts/sync-indexes.ts`
  - Docs: [20-production-readiness.md](./20-production-readiness.md), [21-smoke-test-checklist.md](./21-smoke-test-checklist.md)

- **Task 017 completed:**
  - Admin data cleanup & reset tools with preview + typed confirmation
  - Service: `data-cleanup.service.ts`; scopes: market, imports, campaigns, inbox, leads/pipeline, notifications, operational_all, everything
  - APIs: `/api/admin/data-cleanup/preview`, `/api/admin/data-cleanup/execute`
  - UI: `/settings/data-cleanup` with Arabic danger warnings
  - Audit: admin notification + server log on execute
  - Docs: [19-data-cleanup-reset.md](./19-data-cleanup-reset.md)

- **Task 016 completed:**
  - Settings & Administration module (admin-only)
  - `SystemSetting` singleton model; `settings.service`, `settings-user.service`, `mailbox-settings.service`
  - Role permissions: `src/lib/permissions.ts` (`admin`, `manager`, `sales`, `marketing`, `viewer`)
  - Mailbox encrypted credentials; verify SMTP/IMAP; `mailboxId` on Campaign/EmailMessage
  - APIs: `/api/settings/system`, `/imports`, `/ai`, `/mailboxes`, `/users`
  - UI: `/settings` hub + users, mailboxes, AI, system, imports subpages
  - AI runtime uses DB provider/model overrides via `resolveAIProvider()`
  - Docs: [18-settings-administration.md](./18-settings-administration.md)

- **Task 015 completed:**
  - Internal notifications system (replaces n8n task — no external integrations)
  - Model: `notifications` with priority, status, actionUrl
  - Services: `notification.service`, `automation-rules.service`
  - Non-blocking hooks on inbox, AI, leads, opportunities, tasks, campaigns, email failures, data quality
  - APIs: `/api/notifications`, unread-count, read, read-all, delete
  - UI: topbar bell + dropdown, `/notifications` page, dashboard recent notifications widget

- **Task 014 completed:**
  - Dashboard analytics service aggregating market, imports, campaigns, inbox, AI, leads, pipeline, tasks
  - API: `GET /api/dashboard`
  - UI: `/dashboard` with KPI cards, sections, simple bar charts, recent activities, refresh
  - Components: `kpi-card`, `stat-section`, `simple-bar-list`, `recent-activities`, `dashboard-client`
  - Graceful empty/fallback state when DB unavailable

- **Task 013 completed:**
  - Leads & Pipeline module: lead/opportunity/task services and APIs
  - Classification mapping rules (`src/lib/lead-rules.ts`)
  - Email reply → Lead conversion with duplicate prevention (`emailReplyId` unique)
  - Conversion creates Lead + Opportunity + follow-up Task when appropriate
  - UI: `/leads`, `/leads/new`, `/leads/[id]`, `/pipeline`, `/tasks`, `/opportunities/[id]`
  - Inbox AI panel: **تحويل إلى Lead** with optional force
  - Model updates: `emailReplyId`, `aiClassificationId` on Lead; source refs on Opportunity
  - Activity type `lead_created`; stage probability auto-update on move

- **Task 012 completed:**
  - Provider-agnostic AI layer: `src/server/ai/` (fallback, OpenAI, Gemini/Anthropic placeholders)
  - Email reply classification service and `AiClassification` persistence
  - APIs: `/api/ai/status`, `/api/ai/classify-email-reply`, `/api/ai/classify-email-replies/bulk`, `/api/ai/classify-preview`
  - UI: inbox AI panel, bulk classify on inbox list, `/ai/status` page + sidebar link
  - Env: `AI_PROVIDER`, `OPENAI_MODEL`, `GEMINI_*`, `ANTHROPIC_*` (see `.env.example`)
  - Lead creation from classification explicitly deferred to Task 013

- **Task 011 completed:**
  - IMAP connection/sync foundation with parsed inbox emails
  - New services: `imap.service.ts`, `inbox.service.ts`
  - New APIs: `/api/inbox`, `/api/inbox/[id]`, `/api/inbox/sync`
  - New API: `/api/email/verify-imap`
  - New UI: `/inbox`, `/inbox/[id]` with sync/verify/actions
  - Reply linking to EmailMessage/Campaign/Company/Person where possible
  - `EmailReply` model expanded with IMAP/message metadata fields and dedupe support
  - `email_replied` activities created for linked replies
  - AI classification explicitly deferred to next task

- **Task 010 completed:**
  - SMTP transport service with verification and safe send errors
  - Email rendering service with personalization + tracking injection
  - New services: `email-transport.service.ts`, `email-render.service.ts`, `email-sending.service.ts`
  - New APIs: `/api/campaigns/[id]/send-batch`, `/api/campaigns/[id]/send-one`
  - New APIs: `/api/email/verify-smtp`, `/api/email/test-send`
  - New public tracking APIs: `/api/email-tracking/open/[trackingId]`, `/api/email-tracking/click/[trackingId]`
  - Campaign detail UI send actions and richer recipient status columns (tracking/sent/open/click/error)
  - New protected UI page: `/email-test`
  - Added env controls: `EMAIL_SEND_BATCH_SIZE`, `EMAIL_SEND_DELAY_MS`, `EMAIL_TRACKING_ENABLED`
  - Campaign sending currently synchronous batch-by-batch (queue deferred)

- **Task 009 completed:**
  - Campaigns module preparation layer (no SMTP sending yet)
  - Campaign schema snapshots: `subjectSnapshot`, `templateSnapshot`, `segmentSnapshot`
  - New model: `campaign_recipients` for generated recipients and delivery status foundation
  - Service: `src/server/services/campaign.service.ts`
  - APIs: `/api/campaigns`, `/api/campaigns/[id]`, `/preview-recipients`, `/generate-recipients`, `/recipients`, `/stats`
  - UI: `/campaigns`, `/campaigns/new`, `/campaigns/[id]`, `/campaigns/[id]/edit`
  - Recipient deduplication by email and skipped-reason handling
  - Disabled Send button placeholder for future Email Sending Engine task

- **Task 008 completed:**
  - Email Templates module with variable extraction, validation, and previews
  - Template utilities: `src/lib/template.ts`
  - Service: `src/server/services/email-template.service.ts`
  - APIs: `/api/email-templates`, `/api/email-templates/[id]`, `/api/email-templates/preview`, `/api/email-templates/[id]/preview`
  - UI: `/email-templates`, `/email-templates/new`, `/email-templates/[id]`, `/email-templates/[id]/edit`
  - Sidebar: Email Templates navigation after Segments

- **Task 007 completed:**
  - Segments module for saved targeting filters over imported market data
  - Segment schema update: `targetType`, `description`, `staticIds`, status lifecycle
  - Service: `src/server/services/segment.service.ts`
  - APIs: `/api/segments`, `/api/segments/[id]`, `/api/segments/preview`, `/api/segments/[id]/members`
  - UI: `/segments`, `/segments/new`, `/segments/[id]`, `/segments/[id]/edit`
  - Sidebar: Segments navigation before Campaigns
  - Soft deactivate flow and dynamic preview (count + sample)

- **Task 006 completed:**
  - Market Database module — browse/search/filter over imported market data
  - Service: `src/server/services/market.service.ts` (companies, people, contacts, duplicates, data quality)
  - APIs: `/api/market/companies`, `/api/market/people`, `/api/market/contact-points`, `/api/market/duplicate-reviews`, `/api/market/data-quality-issues`
  - UI: `/market` hub, `/market/companies`, `/market/companies/[id]`, `/market/people`, `/market/contact-points`, `/market/duplicate-reviews`, `/market/data-quality`
  - Sidebar: Market Database section with sublinks
  - Helpers: `src/lib/pagination.ts`, `src/lib/api-response.ts`
  - Decision: Companies Module reframed as Market Database Module (see decisions log)

- **Task 005 completed:**
  - Import system: simple + processed Excel paths
  - Services: excel-parser, import-detector, simple-import, processed-import
  - APIs: `/api/imports/detect`, `preview`, `execute`, `history`
  - Protected UI `/imports` with upload, detect, preview, execute, history
  - `docs/18-import-system.md`, normalization utilities, `src/types/import.ts`
  - Dependencies: `xlsx`, `nanoid`

- **Task 004 completed:**
  - 19 Mongoose models in `src/server/models/` (+ existing `users`)
  - `src/server/models/index.ts` barrel export
  - `src/lib/enums.ts` shared enum constants
  - Import support: `Source`, `ImportLog` (simple/processed), `DataQualityIssue`, `DuplicateReview`
  - CRM: Company, Person, ContactPoint, Lead, Opportunity, Activity, Task
  - Growth: Segment, EmailTemplate, Campaign, EmailMessage, EmailReply, Mailbox
  - AI/automation: AiClassification, AutomationLog, ServiceMatch
  - Schema notes: `errorMessages` (not `errors`), `providerModel` (not `model`) for Mongoose compatibility
  - Full update to `docs/02-database-schema.md`

- **Task 003 completed:**
  - NextAuth.js credentials authentication with JWT sessions
  - `users` Mongoose model (bcrypt `passwordHash`, roles, status)
  - `auth.service.ts` — find user, verify password, bootstrap admin
  - `GET/POST /api/auth/[...nextauth]`, `POST /api/admin/bootstrap` (bootstrap secret header)
  - Middleware — protected pages and APIs; login redirect
  - `/login` (Arabic RTL), `/dashboard` with AppShell (sidebar, sign out)
  - Env: `INITIAL_ADMIN_*`, `BOOTSTRAP_SECRET`
  - Documentation: architecture, API, UI, schema, security, env vars

- **Task 002 completed:**
  - Next.js 16 App Router foundation at repo root (`anwal-growth-platform`)
  - TypeScript, Tailwind CSS v4, ESLint
  - `src/lib/env.ts` — Zod validation for server env (core required; integrations optional)
  - `src/lib/db.ts` — Mongoose connection with dev cache
  - `GET /api/health` — liveness and optional MongoDB status
  - Home page (`/`) — RTL Arabic layout, foundation messaging
  - Project structure: `components/`, `server/services`, `server/repositories`, `types/`
  - Documentation: architecture, API, UI, env vars, deployment, decisions (shadcn deferred)

- **Task 001 completed:**
  - Added [docs/README.md](./README.md) documentation index
  - Added [tasks/README.md](../tasks/README.md) task execution index with status column
  - Added root [`.env.example`](../.env.example) with all planned variables and security comments
  - Synced [10-environment-variables.md](./10-environment-variables.md) to match `.env.example` (purpose, required now, phase)
  - Strengthened [15-cursor-rules.md](./15-cursor-rules.md) with task index, env, API, and page update rules
  - Updated [14-roadmap.md](./14-roadmap.md) — Task 000–001 done, Task 002 next

- **Task 000 completed:**
  - Created documentation structure under `/docs`
  - Created task structure under `/tasks`
  - Added initial product vision and eight capability layers
  - Added project rules in `15-cursor-rules.md`
  - Added roadmap with ten phases
  - Added decisions log with initial architecture choices
  - Added Anwal company context
  - Created `/src` placeholder for application code
  - Added root `README.md` and `.gitignore`

---

## How to write entries

After each task:

```markdown
### Added / Changed / Fixed — Task NNN: short title
- Bullet describing user-visible or doc change
- Reference docs updated
```

Link to the task file in `/tasks` when helpful.