Self-Help APIs for Service Debugging

Overview

Self-help APIs provide operational capabilities for debugging and managing services without requiring code deployments. These endpoints enable cache inspection, queue monitoring, and dynamic configuration changes.

SQS Queue Monitoring

Get Queue Size

Monitor message backlog in SQS queues:

curl --location --request GET \
  'http://<load-balancer>/app/selfhelp/sqs-queue-size?queue_name=prod_device_update_queue' \
  --header 'auth-key: <auth-key>'

Response:

{
  "queue_name": "prod_device_update_queue",
  "approximate_message_count": 1234,
  "approximate_messages_not_visible": 56
}

Cache Management

Get All Cache Names

List available EhCache instances:

curl --location --request GET \
  'http://<load-balancer>/app/eh-cache?cache_name=all' \
  --header 'api-key: <auth-key>'

Get Cache Contents

Retrieve all entries in a specific cache:

# Get all entries in a cache
curl --location --request GET \
  'http://<load-balancer>/app/eh-cache?cache_name=comByCid' \
  --header 'api-key: <auth-key>'

# Get specific cache entry by key
curl --location --request GET \
  'http://<load-balancer>/app/eh-cache?cache_name=comByCid&cache_key=27' \
  --header 'api-key: <auth-key>'

Common Cache Names

Delete Cache Entry

Invalidate a specific cache:

curl --location --request DELETE \
  'http://<instance-ip>:8080/app/eh-cache?cache_name=comById' \
  --header 'api-key: <auth-key>'

Note: Cache deletion should be done on specific instance IPs, not through the load balancer, to ensure all nodes are cleared.

Dynamic Configuration

Get Current Batch Size

curl --location --request GET \
  'http://<load-balancer>/app/utility/v1/updateBatch' \
  --header 'Content-Type: application/x-www-form-urlencoded'

Update Batch Size

Dynamically adjust processing batch size stored in Redis:

curl --location --request POST \
  'http://<load-balancer>/app/utility/v1/updateBatch' \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode 'batch_size=10'

Entity Lookup APIs

Get Entity Details by Component

curl --location --request GET \
  'http://<load-balancer>/app/visitor/v1/entity/internal?component_id=27' \
  --header 'api-key: <auth-key>' \
  --header 'Authorization: Bearer <token>'

Implementation Pattern

Spring Controller Example

@RestController
@RequestMapping("/selfhelp")
public class SelfHelpController {
    
    @Autowired
    private CacheManager cacheManager;
    
    @Autowired
    private AmazonSQS sqsClient;
    
    @GetMapping("/sqs-queue-size")
    public ResponseEntity<?> getQueueSize(
            @RequestParam("queue_name") String queueName,
            @RequestHeader("auth-key") String authKey) {
        
        validateAuthKey(authKey);
        
        GetQueueAttributesRequest request = new GetQueueAttributesRequest()
            .withQueueUrl(getQueueUrl(queueName))
            .withAttributeNames("ApproximateNumberOfMessages",
                              "ApproximateNumberOfMessagesNotVisible");
        
        Map<String, String> attrs = sqsClient
            .getQueueAttributes(request)
            .getAttributes();
            
        return ResponseEntity.ok(Map.of(
            "queue_name", queueName,
            "approximate_message_count", attrs.get("ApproximateNumberOfMessages"),
            "approximate_messages_not_visible", attrs.get("ApproximateNumberOfMessagesNotVisible")
        ));
    }
    
    @GetMapping("/eh-cache")
    public ResponseEntity<?> getCacheContents(
            @RequestParam("cache_name") String cacheName,
            @RequestParam(value = "cache_key", required = false) String cacheKey,
            @RequestHeader("api-key") String authKey) {
        
        validateAuthKey(authKey);
        
        Cache cache = cacheManager.getCache(cacheName);
        if (cache == null) {
            return ResponseEntity.notFound().build();
        }
        
        if (cacheKey != null) {
            return ResponseEntity.ok(cache.get(cacheKey));
        }
        
        // Return all cache entries (implementation-specific)
        return ResponseEntity.ok(getAllCacheEntries(cache));
    }
    
    @DeleteMapping("/eh-cache")
    public ResponseEntity<?> clearCache(
            @RequestParam("cache_name") String cacheName,
            @RequestHeader("api-key") String authKey) {
        
        validateAuthKey(authKey);
        
        Cache cache = cacheManager.getCache(cacheName);
        if (cache != null) {
            cache.clear();
            return ResponseEntity.ok(Map.of("message", "Cache cleared"));
        }
        return ResponseEntity.notFound().build();
    }
}

Security Considerations

1. Authentication - All self-help APIs require API key headers
2. Network Access - Restrict access to internal networks only
3. Audit Logging - Log all cache modifications and configuration changes
4. Rate Limiting - Prevent abuse of diagnostic endpoints

Best Practices

1. Use for Debugging Only - Don’t rely on these APIs for normal operations
2. Clear All Nodes - When invalidating caches, ensure all instances are updated
3. Monitor After Changes - Watch metrics after dynamic configuration changes
4. Document Changes - Log who made changes and why in operational runbooks