Issue #136: Git Provider Integration

Summary

Issue #136 requested the implementation of Git provider integration to enable automated invoice generation from git commits. The implementation includes connecting to GitHub, GitLab, and Bitbucket, syncing repositories, validating tokens, and linking repositories to projects for project-based billing.

Status: 🚧 IN PROGRESS - Core provider management and token validation features are implemented. Additional features are still being developed.

What Was Implemented

1. Git Provider Management

Supported Providers: - GitHub (personal access tokens with repo scope) - GitLab (personal access tokens) - Bitbucket (app passwords)

Key Features: - ✅ Multi-provider support (connect multiple accounts per provider type) - ✅ Token validation with automatic error recovery - ✅ Repository synchronization with project linking - ✅ Active/inactive provider status management - ✅ Token expiration handling with modal-based token update - ✅ Event bus integration for all operations - ✅ Automatic repository discovery and selection

2. User Interface

Provider List Page (/member/feature/git_connect)

Components: - Provider connection modal with provider type selection - Token input with validation - Provider list with active/inactive badges - Token validation button per provider - Delete provider action - View repositories button (centered outlined button with icon)

Features: - GitHub token URL includes current date to avoid name collisions - Format: MeisterBill Git Connect YYYY-MM-DD - Example: MeisterBill Git Connect 2025-01-14 - Automatic token validation on connection - Edit modal auto-opens on validation failure - Provider automatically deactivated on token validation failure

Provider Detail Page (/member/feature/git_connect/:id)

Components: - Repository list with pagination - Repository sync button (moved from list page) - Select repositories button - Repository metadata (name, URL, default branch)

Features: - Sync repositories from git provider - Link repositories to projects - Display sync status (total repos, new repos discovered)

3. Token Recovery Flow

Problem Solved: When a token validation fails (e.g., token expired or revoked at provider), there was no way to update the token without navigating to settings and manually editing the provider.

Solution Implemented: 1. Click "Validate Token" on provider card 2. If validation fails: - Provider is_active flag automatically set to false - Badge changes from "Active" to "Inactive" - Edit modal opens automatically with pre-filled provider data - Warning alert explains token validation failure - User enters new token - Submit updates provider and reactivates (is_active = true)

User Experience: - No navigation required - stay on same page - Visual feedback with badge color change - Clear explanation of what went wrong - Seamless token update process - Automatic reactivation on successful update

4. Event Bus Architecture

All git provider operations emit events through the centralized event bus instead of showing toasts directly. This decouples UI feedback from business logic and enables future integrations (Discord notifications, audit logs, WebSocket updates, etc.).

Event Schema (schemas/src/EventSchema.ts)

Nine new git provider events were added:

// Connection events
export const EVENT_GIT_PROVIDER_CONNECTED = "git_provider.connected";
export const EVENT_GIT_PROVIDER_CONNECTED_FAILED = "git_provider.connected.failed";

// Update events
export const EVENT_GIT_PROVIDER_UPDATED = "git_provider.updated";
export const EVENT_GIT_PROVIDER_UPDATED_FAILED = "git_provider.updated.failed";

// Deletion events
export const EVENT_GIT_PROVIDER_DELETED = "git_provider.deleted";

// Validation events
export const EVENT_GIT_PROVIDER_TOKEN_VALIDATED = "git_provider.token.validated";
export const EVENT_GIT_PROVIDER_TOKEN_VALIDATION_FAILED = "git_provider.token.validation.failed";

// Sync events
export const EVENT_GIT_PROVIDER_REPOS_SYNCED = "git_provider.repos.synced";
export const EVENT_GIT_PROVIDER_REPOS_SYNC_FAILED = "git_provider.repos.sync.failed";

Each event includes: - type: Event name constant - data: Event-specific payload (provider ID, type, error message, sync counts, etc.)

Toast Subscriber (apps/web/plugins/05.EventBusSubscribers.ts)

All git provider events are handled by the centralized toast subscriber:

// Success events → Success toasts
eventBus.on(EVENT_GIT_PROVIDER_CONNECTED, () => {
  addToast({
    title: nuxtApp.$i18n.t("message.git_provider_connected"),
    color: "success",
  });
});

// Failure events → Error toasts with details
eventBus.on(EVENT_GIT_PROVIDER_CONNECTED_FAILED, (event) => {
  addToast({
    title: nuxtApp.$i18n.t("error.failed_to_connect_git_provider"),
    description: event.data.error,
    color: "error",
  });
});

Benefits: - ✅ Decoupled UI feedback from business logic - ✅ Single source of truth for toast messages - ✅ Easy to add new notification channels (Discord, WebSocket) - ✅ Centralized i18n for all user feedback - ✅ Consistent error handling across all operations

5. UI/UX Improvements

Smart Button Placement: - "View Repositories" converted from plain link to centered outlined button with folder-git icon - "Sync Repositories" button moved from provider list page to repository detail page - Logical grouping: sync button appears next to "Select Repositories" button - Better visual hierarchy and clearer call-to-action

Visual Feedback: - Active/Inactive badges with color coding (success/neutral) - Loading spinners during validation and sync operations - Toast notifications for all operations (via event bus) - Icon-based provider type display (GitHub, GitLab, Bitbucket icons) - Warning alerts in edit modal when token validation fails

6. Database Schema

git_providers Table:

CREATE TABLE git_providers (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  service_provider_id UUID NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE,
  provider_type VARCHAR(50) NOT NULL, -- 'github', 'gitlab', 'bitbucket'
  access_token TEXT NOT NULL, -- Encrypted
  token_description TEXT,
  is_active BOOLEAN DEFAULT true,
  created_at TIMESTAMP WITH TIME ZONE DEFAULT now(),
  updated_at TIMESTAMP WITH TIME ZONE DEFAULT now()
);

git_repositories Table:

CREATE TABLE git_repositories (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  git_provider_id UUID NOT NULL REFERENCES git_providers(id) ON DELETE CASCADE,
  project_id UUID REFERENCES projects(id) ON DELETE SET NULL,
  repository_name VARCHAR(255) NOT NULL,
  repository_url TEXT NOT NULL,
  default_branch VARCHAR(100) DEFAULT 'main',
  last_synced_at TIMESTAMP WITH TIME ZONE,
  created_at TIMESTAMP WITH TIME ZONE DEFAULT now(),
  updated_at TIMESTAMP WITH TIME ZONE DEFAULT now()
);

7. API Endpoints

Provider Management: - GET /git-providers - List all connected git providers - POST /git-providers - Connect new git provider - PUT /git-providers/:id - Update provider details (token, description, status) - DELETE /git-providers/:id - Remove git provider - POST /git-providers/:id/validate - Validate provider token - POST /git-providers/:id/sync-repositories - Sync repositories from provider

Response Format:

// Validation success
{
  success: true
}

// Validation failure
{
  success: false,
  error: "Invalid token or insufficient permissions"
}

// Sync success
{
  success: true,
  total: 45,        // Total repositories at provider
  new: 3            // Newly discovered repositories
}

8. Implementation Files

Frontend: - apps/web/pages/member/feature/git_connect/index.vue - Provider list and management - apps/web/pages/member/feature/git_connect/[id].vue - Repository detail and sync - apps/web/composables/useGitProvider.ts - Git provider API client - apps/web/plugins/05.EventBusSubscribers.ts - Toast event handlers - apps/web/i18n/locales/en.json - i18n translations

Backend: - apps/api/src/routes/git_providers.ts - API endpoints - apps/api/src/routes/git_repositories.ts - Repository endpoints

Shared: - schemas/src/EventSchema.ts - Event definitions - schemas/src/GitProviderSchema.ts - Zod schemas for validation

9. Workflow Integration

Connection to Project-Based Billing:

1. Connect Provider → User authenticates with GitHub/GitLab/Bitbucket
2. Sync Repositories → Fetch all accessible repositories from provider
3. Link to Project → Associate repositories with billing projects
4. Sync Commits → Fetch commits from linked repositories (see Project-Based Architecture)
5. Create Work Units → Parse commits and group by issue/PR numbers
6. Generate Invoice → Convert work units to invoice line items

The git provider integration is the foundation for automated invoice generation from git activity.

10. Security Considerations

Token Storage: - Access tokens stored encrypted in database - Tokens never exposed in API responses - Tokens never sent to frontend (only validation status)

Token Validation: - Server-side validation only (prevents client bypass) - Validates token permissions and scopes - Checks repository access permissions

Scoping: - Provider credentials scoped to individual service providers - Users can only see/manage their own providers - Repository access respects provider permissions

Token Descriptions: - Help users identify tokens in provider settings - Useful for finding and revoking expired tokens - Includes date in GitHub token name to avoid collisions

Benefits

For Service Providers:

• ✅ Automated Billing - Generate invoices from git commits
• ✅ Multi-Provider Support - Connect multiple GitHub/GitLab/Bitbucket accounts
• ✅ Easy Token Management - Update tokens without losing repository links
• ✅ Visual Status Tracking - Clear active/inactive status indicators
• ✅ Self-Service Recovery - Fix expired tokens without support

For Development:

• ✅ Event-Driven Architecture - Decoupled, extensible event system
• ✅ Type-Safe Events - Full TypeScript support with Zod validation
• ✅ Centralized Feedback - Single source of truth for toast messages
• ✅ Easy to Extend - Add new providers or notification channels

For Users:

• ✅ Clear Workflow - Intuitive connect → sync → link process
• ✅ Instant Feedback - Loading states and toast notifications
• ✅ Error Recovery - Auto-modal for token updates on failure
• ✅ Visual Consistency - Icons, badges, and color-coded status

User Stories Covered

Story 1: Connect Git Provider

As a service provider I want to connect my GitHub account So that I can sync my repositories for billing

Acceptance Criteria: - ✅ Select provider type (GitHub, GitLab, Bitbucket) - ✅ Generate token with instructions and date-stamped name - ✅ Enter token and optional description - ✅ Token validated automatically on connection - ✅ Success/failure feedback via toast notification

Story 2: Validate Token

As a service provider I want to validate my git provider token So that I can ensure it's still active

Acceptance Criteria: - ✅ Click "Validate Token" button on provider card - ✅ Loading spinner during validation - ✅ Badge updates to "Active" or "Inactive" based on result - ✅ Toast notification shows success or error message

Story 3: Recover from Expired Token

As a service provider I want to easily update my token when it expires So that I can continue syncing repositories without manual navigation

Acceptance Criteria: - ✅ Token validation failure automatically opens edit modal - ✅ Modal pre-filled with current provider data - ✅ Warning alert explains validation failure - ✅ New token input field - ✅ Submit updates token and reactivates provider - ✅ No page navigation required

Story 4: Sync Repositories

As a service provider I want to sync repositories from my git provider So that I can link them to billing projects

Acceptance Criteria: - ✅ Click "Sync Repositories" button on provider detail page - ✅ Loading spinner during sync operation - ✅ Toast shows total and new repository counts - ✅ Repository list updates automatically after sync - ✅ Error handling with descriptive messages

Story 5: Link Repositories to Projects

As a service provider I want to select which repositories to link to a project So that I can generate invoices from those repositories' commits

Acceptance Criteria: - ✅ Click "Select Repositories" button - ✅ Repository selection modal - ✅ Link repositories to project - ✅ Multiple repositories can be linked to same project

Testing Recommendations

Unit Tests

• ✅ Event schema validation (Zod schemas)
• ✅ Event bus emission and subscription
• ✅ Toast subscriber event handling
• ⏳ Git provider composable methods (TODO)
• ⏳ Token validation logic (TODO)

Integration Tests

• ⏳ Provider connection flow (TODO)
• ⏳ Token validation with mock API (TODO)
• ⏳ Repository sync with mock provider API (TODO)
• ⏳ Edit modal auto-open on validation failure (TODO)

E2E Tests

• ⏳ Complete provider connection workflow (TODO)
• ⏳ Token expiration and recovery flow (TODO)
• ⏳ Repository sync and project linking (TODO)

Future Enhancements

Potential Improvements:

1. Webhook Integration - Real-time commit notifications from git providers
2. OAuth Flow - Replace personal tokens with OAuth for better UX
3. Token Refresh - Automatic token renewal for OAuth providers
4. Repository Filters - Filter repos by organization, language, or activity
5. Batch Operations - Bulk link/unlink repositories to projects
6. Provider Health Dashboard - Track token status, sync frequency, API rate limits
7. Additional Providers - Azure DevOps, Gitea, self-hosted GitLab
8. Commit Preview - Show recent commits before syncing to project

Related Documentation

• Project Structure - How git repositories integrate with projects
• Event Bus Architecture - Event system documentation
• Features Catalogue - Complete feature list

Current Status

✅ Completed Features

• Git provider connection (GitHub, GitLab, Bitbucket)
• Token validation with auto-recovery flow
• Repository synchronization
• Event bus integration for all operations
• UI improvements (button placement, icons, badges)
• Token expiration handling with modal-based token update
• Active/inactive provider status management

🚧 In Progress

• Repository selection and project linking UI
• Commit syncing from repositories
• Work unit creation from commits
• Invoice generation from work units

📋 Pending

• Webhook integration for real-time updates
• OAuth flow (to replace personal tokens)
• Additional provider support (Azure DevOps, Gitea)
• Batch operations for repository management

Changelog

Latest Changes (2025-01-14)

• ✅ Token validation with auto-recovery modal implemented
• ✅ UI improvements for git provider management
• ✅ Event bus integration for all git provider operations
• ✅ Documentation created in docs/issues/issue-0136.md

Commits:

• fce4de0 - #136 ✨ feat: Implement git provider token validation with auto-recovery modal
• 75d974d - #136 ✨ feat: Improve git provider UI and repository sync placement
• 3d5f1aa - #136 📝 docs: Add comprehensive Git Provider Integration documentation