specs/012-credit-per-item/design.md
Rama main · réplica de lectura del repositorio original.
# Credit-Per-Item: Design Document
## Overview
This feature implements credit enforcement across the news processing pipeline. Credits are deducted when news items undergo AI processing, and all AI-dependent operations are blocked when credits reach zero.
**Key Design Principles:**
- Block at scheduler level for efficiency (batch queries, not per-item checks)
- Defense-in-depth: safety net in `TrackedAiService` as last resort
- Non-destructive blocking: items stay in current status, resume when credits replenished
- Single credit deduction point: `ProcessingWorker` after successful AI enrichment
## Architecture
```
┌─────────────────────────────────────────────────────────────────────────────────┐
│ CREDIT ENFORCEMENT LAYERS │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ LAYER 1: SCHEDULER BLOCKING (Primary) │
│ ┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │ FeedSchedulerJob │ │ProcessingSchedulerJob│ │BriefingSchedulerJob│ │
│ │ ─────────────────── │ │ ─────────────────── │ │ ─────────────────── │ │
│ │ Skip sources for │ │ Skip items for │ │ Skip briefings for │ │
│ │ users with 0 credits│ │ users with 0 credits│ │ users with 0 credits│ │
│ └─────────────────────┘ └─────────────────────┘ └─────────────────────┘ │
│ │
│ LAYER 2: LISTENER BLOCKING │
│ ┌─────────────────────┐ │
│ │EmailIngestionListener│ │
│ │ ─────────────────── │ │
│ │ Check credits before │ │
│ │ AI extraction │ │
│ └─────────────────────┘ │
│ │
│ LAYER 3: API BLOCKING │
│ ┌─────────────────────┐ │
│ │DayBriefController │ │
│ │ ─────────────────── │ │
│ │ Check credits before │ │
│ │ manual execution │ │
│ └─────────────────────┘ │
│ │
│ LAYER 4: SAFETY NET (Defense-in-Depth) │
│ ┌─────────────────────┐ │
│ │ TrackedAiService │ │
│ │ ─────────────────── │ │
│ │ Throw exception if │ │
│ │ AI call with 0 credit│ │
│ └─────────────────────┘ │
│ │
│ CREDIT DEDUCTION (existing SubscriptionService) │
│ ┌─────────────────────┐ │
│ │ ProcessingWorker │ ──> SubscriptionService.useCredits() ──> CreditUsageLog │
│ └─────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────────┘
```
## Components and Interfaces
### 1. SubscriptionService (Extended)
Extend existing service with additional methods for credit checking.
**File:** `backend/src/main/java/be/transcode/morningdeck/server/core/service/SubscriptionService.java`
**Existing methods (no changes):**
- `useCredits(User user, int credits)` - deducts credits and logs to `CreditUsageLog`
- `createSubscription()`, `upgradePlan()`, `checkAndRenewSubscriptions()`
**New methods to add:**
```java
/**
* Check if user has any credits (> 0).
* @param userId User ID
* @return true if user has at least 1 credit
*/
public boolean hasCredits(UUID userId) {
return subscriptionRepository.findCreditsBalanceByUserId(userId)
.map(balance -> balance > 0)
.orElse(false);
}
/**
* Get credit balance for user.
* @return credit balance or 0 if subscription not found
*/
public int getCreditsBalance(UUID userId) {
return subscriptionRepository.findCreditsBalanceByUserId(userId).orElse(0);
}
/**
* Get set of user IDs that have credits > 0.
* Used by schedulers for efficient batch filtering.
*/
public Set<UUID> getUserIdsWithCredits() {
return subscriptionRepository.findUserIdsWithCredits();
}
/**
* Overload of useCredits that accepts userId instead of User entity.
* Also triggers no-credits notification when balance hits zero.
*/
@Transactional
public boolean useCredits(UUID userId, int credits) {
User user = userRepository.findById(userId)
.orElseThrow(() -> new ResourceNotFoundException("User not found"));
boolean success = useCredits(user, credits);
// Send notification if balance just hit zero
if (success) {
Subscription sub = user.getSubscription();
if (sub.getCreditsBalance() == 0) {
sendNoCreditsNotification(user);
}
}
return success;
}
private void sendNoCreditsNotification(User user) {
try {
emailService.sendNoCreditsEmail(user.getEmail(), user.getName(), domain);
log.info("Sent no-credits notification to user {}", user.getId());
} catch (Exception e) {
log.error("Failed to send no-credits notification to user {}: {}", user.getId(), e.getMessage());
}
}
```
### 2. InsufficientCreditsException (New)
**File:** `backend/src/main/java/be/transcode/morningdeck/server/core/exception/InsufficientCreditsException.java`
```java
public class InsufficientCreditsException extends RuntimeException {
private final UUID userId;
private final int creditsRequired;
private final int creditsAvailable;
public InsufficientCreditsException(UUID userId, int required, int available) {
super(String.format("Insufficient credits for user %s: required=%d, available=%d",
userId, required, available));
this.userId = userId;
this.creditsRequired = required;
this.creditsAvailable = available;
}
}
```
### 3. Modified Components
#### 3.1 TrackedAiService (Safety Net)
Add credit check at the start of `trackCall()` method:
```java
private <T> T trackCall(AiFeature feature, Supplier<AiCallResult<T>> call) {
UUID userId = AiUsageContext.getUserId();
// Safety net: verify credits if user context is set
if (userId != null) {
int balance = subscriptionService.getCreditsBalance(userId);
if (balance <= 0) {
log.error("AI call attempted with zero credits: userId={} feature={}", userId, feature);
throw new InsufficientCreditsException(userId, 1, balance);
}
}
// ... existing tracking logic
}
```
#### 3.2 ProcessingWorker (Credit Deduction)
Add credit deduction after successful AI processing:
```java
private void doProcess(NewsItem item) {
// ... existing content preparation ...
UUID userId = item.getSource().getDayBrief().getUserId();
try {
AiUsageContext.setUserId(userId);
// Enrich and score
EnrichmentWithScoreResult result = aiService.enrichWithScore(...);
// Apply results to item
item.setSummary(result.summary());
// ... set tags, score, etc.
// Deduct credit after successful processing
boolean deducted = subscriptionService.useCredits(userId, 1);
if (!deducted) {
// This should not happen due to scheduler-level checks
log.error("Failed to deduct credit after AI processing: userId={}", userId);
throw new InsufficientCreditsException(userId, 1, 0);
}
} finally {
AiUsageContext.clear();
}
}
```
#### 3.3 ProcessingSchedulerJob
Filter out items for users with zero credits:
```java
@Scheduled(fixedRateString = "${application.jobs.news-processing.interval:60000}")
public void scheduleProcessing() {
if (!processingQueue.canAccept()) {
log.warn("Queue full, skipping scheduling cycle");
return;
}
// Get users with available credits
Set<UUID> usersWithCredits = subscriptionService.getUserIdsWithCredits();
if (usersWithCredits.isEmpty()) {
log.debug("No users with credits available");
return;
}
// Find items for users with credits
List<NewsItem> candidates = newsItemRepository.findItemsForProcessingByUserIds(
usersWithCredits, PageRequest.of(0, batchSize));
// ... rest of scheduling logic
}
```
#### 3.4 FeedSchedulerJob
Filter out sources for users with zero credits:
```java
@Scheduled(fixedRateString = "${application.jobs.feed-scheduling.interval:60000}")
@Transactional
public void scheduleFeeds() {
if (!fetchQueue.canAccept()) {
return;
}
// Get users with credits
Set<UUID> usersWithCredits = subscriptionService.getUserIdsWithCredits();
// Filter candidates by users with credits
List<Source> dueForRefresh = candidates.stream()
.filter(s -> isDueForRefresh(s, now))
.filter(s -> usersWithCredits.contains(s.getDayBrief().getUserId()))
.limit(batchSize)
.toList();
// ... rest of scheduling logic
}
```
#### 3.5 BriefingSchedulerJob
Filter out briefings for users with zero credits:
```java
@Scheduled(fixedRateString = "${application.jobs.briefing-execution.interval:60000}")
public void scheduleBriefings() {
if (!briefingQueue.canAccept()) {
return;
}
// Get users with credits
Set<UUID> usersWithCredits = subscriptionService.getUserIdsWithCredits();
List<DayBrief> dueBriefings = candidateBriefings.stream()
.filter(briefing -> isBriefingDue(briefing, nowUtc))
.filter(briefing -> usersWithCredits.contains(briefing.getUserId()))
.toList();
// ... rest of scheduling logic
}
```
#### 3.6 EmailIngestionListener
Check credits before AI extraction:
```java
private void processEmail(Source source, EmailMessage email) {
UUID userId = source.getDayBrief().getUserId();
// Check credits before AI extraction
if (!subscriptionService.hasCredits(userId)) {
log.warn("Email received but user {} has no credits. Stored but not processed: messageId={}",
userId, email.getMessageId());
markRawEmailAsUnprocessed(source, email);
return;
}
try {
AiUsageContext.setUserId(userId);
List<ExtractedNewsItem> items = aiService.extractFromEmail(email.getSubject(), email.getContent());
// ... create news items
} finally {
AiUsageContext.clear();
}
}
```
#### 3.7 DayBriefController
Check credits for manual briefing execution:
```java
@PostMapping("/{id}/execute")
public ResponseEntity<DailyReportDTO> executeBriefing(
@AuthenticationPrincipal UserDetails userDetails,
@PathVariable UUID id) {
User user = userService.getInternalUserByUsername(userDetails.getUsername());
// Check credits
if (!subscriptionService.hasCredits(user.getId())) {
throw new InsufficientCreditsException(user.getId(), 1, 0);
}
DayBrief dayBrief = dayBriefService.getDayBriefEntity(user.getId(), id);
DailyReportDTO report = briefingWorker.executeManually(dayBrief);
return ResponseEntity.ok(report);
}
```
#### 3.8 GlobalExceptionHandler
Handle `InsufficientCreditsException`:
```java
@ExceptionHandler(InsufficientCreditsException.class)
public ResponseEntity<ApiError> handleInsufficientCredits(
InsufficientCreditsException ex, HttpServletRequest request) {
return buildErrorResponse(HttpStatus.PAYMENT_REQUIRED, ex.getMessage(), request);
}
```
## Data Models
### Existing Models (No Changes)
- **Subscription** - `creditsBalance`, `monthlyCredits` fields already exist
- **CreditUsageLog** - Already tracks credit usage with `user`, `subscription`, `creditsUsed`, `usedAt`
### Modified Models
#### RawEmail (Add Processing Status)
For REQ-9 (optional email reprocessing), add a status field:
```java
@Entity
@Table(name = "raw_emails")
public class RawEmail {
// ... existing fields ...
@Enumerated(EnumType.STRING)
@Column(name = "processing_status", nullable = false)
@Builder.Default
private RawEmailStatus processingStatus = RawEmailStatus.PENDING;
}
public enum RawEmailStatus {
PENDING, // Awaiting processing
PROCESSED, // Successfully processed
SKIPPED // Skipped due to insufficient credits
}
```
**Migration:** `V__add_raw_email_status.sql`
```sql
ALTER TABLE raw_emails ADD COLUMN processing_status VARCHAR(20) DEFAULT 'PROCESSED' NOT NULL;
CREATE INDEX idx_raw_emails_status ON raw_emails(processing_status) WHERE processing_status = 'SKIPPED';
```
### New Repository Methods
#### SubscriptionRepository
```java
@Query("SELECT s.user.id FROM Subscription s WHERE s.creditsBalance > 0")
Set<UUID> findUserIdsWithCredits();
@Query("SELECT s.creditsBalance FROM Subscription s WHERE s.user.id = :userId")
Optional<Integer> findCreditsBalanceByUserId(@Param("userId") UUID userId);
```
#### NewsItemRepository
```java
@Query("""
SELECT n FROM NewsItem n
WHERE n.status = be.transcode.morningdeck.server.core.model.NewsItemStatus.NEW
AND n.source.dayBrief.userId IN :userIds
ORDER BY n.createdAt ASC
""")
List<NewsItem> findItemsForProcessingByUserIds(
@Param("userIds") Set<UUID> userIds,
Pageable pageable);
```
#### RawEmailRepository (for REQ-9)
```java
List<RawEmail> findByProcessingStatus(RawEmailStatus status);
@Query("SELECT r FROM RawEmail r WHERE r.processingStatus = 'SKIPPED' AND r.source.dayBrief.userId = :userId")
List<RawEmail> findSkippedByUserId(@Param("userId") UUID userId);
```
## Error Handling
| Exception | HTTP Status | Trigger |
|-----------|-------------|---------|
| `InsufficientCreditsException` | 402 Payment Required | Manual briefing execution, safety net |
| N/A (silent skip) | N/A | Scheduler skips items/sources/briefings |
**Logging Strategy:**
- `INFO`: Items/sources/briefings skipped due to zero credits
- `WARN`: Email stored but not processed due to zero credits
- `ERROR`: AI call attempted despite zero credits (safety net triggered)
## Testing Strategy
### Unit Tests
| Component | Test Cases |
|-----------|------------|
| `SubscriptionService` | hasCredits(), useCredits(UUID), getUserIdsWithCredits(), notification trigger |
| `ProcessingWorker` | Credit deduction after success, handling deduction failure |
| `TrackedAiService` | Safety net throws exception when credits = 0 |
### Integration Tests
| Test Class | Scenarios |
|------------|-----------|
| `CreditEnforcementIT` | End-to-end: create items, process, verify credit deduction |
| `SchedulerCreditIT` | Scheduler skips users with zero credits |
| `EmailIngestionCreditIT` | Email stored but not processed when credits = 0 |
| `ManualExecutionCreditIT` | Manual briefing execution returns 402 when credits = 0 |
### Test Data Setup
```java
// User with credits
User userWithCredits = createUserWithSubscription(SubscriptionPlan.FREE); // 900 credits
// User without credits
User userNoCredits = createUserWithSubscription(SubscriptionPlan.FREE);
userNoCredits.getSubscription().setCreditsBalance(0);
```
## Performance Considerations
### Batch Credit Checking
Instead of checking credits per-item in schedulers:
```java
// BAD: N+1 queries
for (NewsItem item : candidates) {
UUID userId = item.getSource().getDayBrief().getUserId();
if (subscriptionService.hasCredits(userId)) { // Query per item
enqueue(item);
}
}
// GOOD: Single query for all user IDs with credits
Set<UUID> usersWithCredits = subscriptionService.getUserIdsWithCredits(); // One query
for (NewsItem item : candidates) {
UUID userId = item.getSource().getDayBrief().getUserId();
if (usersWithCredits.contains(userId)) { // In-memory lookup
enqueue(item);
}
}
```
### Index Considerations
Existing indexes should suffice. The `subscriptions.credits_balance` column may benefit from a partial index:
```sql
CREATE INDEX idx_subscriptions_has_credits ON subscriptions(user_id) WHERE credits_balance > 0;
```
## Security Considerations
### Race Condition Prevention
Credit deduction uses optimistic locking via `@Version`:
```java
@Entity
public class Subscription {
@Version
private Long version; // Already exists in JPA
}
```
In `SubscriptionService.useCredits(UUID, int)`:
```java
@Transactional
public boolean useCredits(UUID userId, int credits) {
Subscription sub = subscriptionRepository.findByUserId(userId)
.orElseThrow();
if (sub.getCreditsBalance() < credits) {
return false;
}
sub.setCreditsBalance(sub.getCreditsBalance() - credits);
subscriptionRepository.save(sub); // OptimisticLockException if concurrent update
// Log usage
creditUsageLogRepository.save(CreditUsageLog.builder()
.user(sub.getUser())
.subscription(sub)
.creditsUsed(credits)
.build());
// Check if balance hit zero - send notification
if (sub.getCreditsBalance() == 0) {
sendNoCreditsNotification(sub.getUser());
}
return true;
}
```
### Input Validation
- Credit deduction only accepts positive values
- User ID validation via existing authentication
## Monitoring and Observability
### Metrics
| Metric | Type | Description |
|--------|------|-------------|
| `credits.deducted.total` | Counter | Total credits deducted |
| `credits.deducted.user` | Counter (tagged by user_id) | Credits per user |
| `credits.exhausted.total` | Counter | Count of users hitting zero credits |
| `scheduler.skipped.no_credits` | Counter | Items/sources/briefings skipped |
| `ai.blocked.no_credits` | Counter | Safety net triggers |
### Logging
Structured log fields for credit operations:
```json
{
"event": "credit_deducted",
"userId": "uuid",
"creditsDeducted": 1,
"newBalance": 899,
"newsItemId": "uuid"
}
```
### Alerts
| Alert | Condition | Severity |
|-------|-----------|----------|
| Safety net triggered | `ai.blocked.no_credits` > 0 | ERROR (indicates bug) |
| High credit exhaustion rate | `credits.exhausted.total` spike | WARN |
## Implementation Order
1. **Phase 1: Core Infrastructure**
- Create `InsufficientCreditsException`
- Extend `SubscriptionService` with new methods (hasCredits, getUserIdsWithCredits, useCredits overload)
- Add new repository methods to `SubscriptionRepository`
- Add handler to `GlobalExceptionHandler`
2. **Phase 2: Credit Deduction**
- Modify `ProcessingWorker` to deduct credits
- Add no-credits notification trigger
3. **Phase 3: Scheduler Blocking**
- Modify `ProcessingSchedulerJob`
- Modify `FeedSchedulerJob`
- Modify `BriefingSchedulerJob`
4. **Phase 4: API and Listener Blocking**
- Modify `DayBriefController.executeBriefing()`
- Modify `EmailIngestionListener`
5. **Phase 5: Safety Net**
- Modify `TrackedAiService`
6. **Phase 6: Email Reprocessing (Optional)**
- Add `RawEmailStatus` enum and migration
- Implement reprocessing job