Python Cloud Config: Accessing Spring Cloud Config from Python Services

Overview

When running a polyglot microservices architecture with Spring Cloud Config as the centralized configuration server, Python services need a way to fetch configurations. This guide covers implementing a Python client that works seamlessly with existing Spring Cloud Config infrastructure.

Why Cloud Config for Python?

The Challenge

• Spring Cloud Config works great for Java services
• No equivalent Python package for remote config + Vault integration
• Existing libraries like spring-config-client and config-client are not well maintained
• Need minimal code changes from existing python-decouple usage

The Solution

A custom library (pyremoteconfig) that:

• Acts as a drop-in replacement for python-decouple
• Connects to Spring Cloud Config Server
• Supports HashiCorp Vault for secrets
• Requires minimal code changes

Architecture

┌───────────────────────────────────────────────────────────────────┐
│                     Configuration Flow                             │
├───────────────────────────────────────────────────────────────────┤
│                                                                    │
│   Git Repository                Spring Cloud                      │
│   (Config Files)                Config Server                      │
│         │                            │                             │
│         │  Pull configs              │  Serve configs              │
│         └────────────────────────────┼──────────────────┐          │
│                                      │                  │          │
│                                      │                  │          │
│                    ┌─────────────────┼─────────────┐    │          │
│                    │                 │             │    │          │
│                    ▼                 ▼             ▼    ▼          │
│              ┌─────────┐       ┌─────────┐   ┌─────────────┐      │
│              │  Java   │       │  Java   │   │   Python    │      │
│              │ Service │       │ Service │   │   Service   │      │
│              │  (A)    │       │  (B)    │   │             │      │
│              └─────────┘       └─────────┘   └─────────────┘      │
│                                                     │              │
│                                              pyremoteconfig        │
│                                                library             │
└───────────────────────────────────────────────────────────────────┘

Server-Side Setup

Create Service Configuration

In your config repository, create a folder for the Python service:

remote-configs/
├── service-a/
│   ├── application.yml          # Default profile
│   ├── application-dev.yml      # Development profile
│   ├── application-prod.yml     # Production profile
│   └── application-stage.yml     # Stage profile
├── service-b/
│   └── ...
└── python-service/
    ├── application.yml
    ├── application-dev.yml
    └── application-prod.yml

Configuration File Structure

# python-service/application.yml (default)
database:
  host: localhost
  port: 5432
  name: myapp

redis:
  host: localhost
  port: 6379

logging:
  level: INFO

# python-service/application-prod.yml
database:
  host: prod-db.internal
  port: 5432
  name: myapp_prod

redis:
  host: prod-redis.internal
  port: 6379

logging:
  level: WARN

Client-Side Setup

Installation

Create environment file for credentials:

# .env file
export RCUSER=bitbucket_username
export RCPASS=bitbucket_app_password

Add to requirements.txt:

# requirements.txt
flask>=2.0
gunicorn>=20.0
# Custom config client
git+https://${RCUSER}:${RCPASS}@bitbucket.org/yourorg/remote-config-python.git

Install in Different Environments

Regular Python:

source .env && pip install -r requirements.txt

Docker:

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt .env ./
RUN . ./.env && pip install -r requirements.txt

COPY . .
CMD ["python", "app.py"]

Bootstrap Configuration

Create bootstrap.yml in your project root:

# bootstrap.yml
python:
  application:
    name: python-service
  profiles: dev
  cloud:
    config:
      label: master
      uri: http://config-server.internal:8888
      token: vault-token-xxx-xxx

---
# Production profile override
python:
  profiles: prod
  cloud:
    config:
      label: master
      token: vault-token-prod-xxx
      uri: http://config-server.internal:8888

Configuration Priority

Values in later YAML documents override earlier ones:

# Document 1 (default)
python:
  application:
    name: python-service    # ← Used (not overridden)
  profiles: dev             # ← Overridden by doc 2
  cloud:
    config:
      label: develop        # ← Overridden by doc 2
      uri: http://localhost:8888  # ← Overridden by doc 2

---
# Document 2 (prod override)
python:
  profiles: prod            # ← Wins
  cloud:
    config:
      label: master         # ← Wins
      uri: http://prod-config:8888  # ← Wins

Code Migration

Before (using python-decouple)

# config.py
from decouple import config

DB_USERNAME = config("DB_USERNAME")
DB_HOST = config("DB_HOST", default="192.168.0.10")
DB_TIMEOUT = config("DB_TIMEOUT", default=10, cast=int)
DB_DEBUG = config("DB_DEBUG", default=False, cast=bool)

After (using remote config)

# config.py
# Just change the import!
from pyremoteconfig import config

DB_USERNAME = config("DB_USERNAME")
DB_HOST = config("DB_HOST", default="192.168.0.10")
DB_TIMEOUT = config("DB_TIMEOUT", default=10, cast=int)
DB_DEBUG = config("DB_DEBUG", default=False, cast=bool)

That’s it! The API is identical to decouple.

Running the Application

Select Profile at Runtime

# Development
export BOOTSTRAP_PROFILE=dev && python app.py

# Production
export BOOTSTRAP_PROFILE=prod && python app.py

# With Gunicorn
export BOOTSTRAP_PROFILE=prod && gunicorn -b 0.0.0.0:8000 app:app

Docker Compose

# docker-compose.yml
version: '3.8'
services:
  python-service:
    build: .
    environment:
      - BOOTSTRAP_PROFILE=dev
    ports:
      - "8000:8000"

Kubernetes

# deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: python-service
spec:
  template:
    spec:
      containers:
      - name: app
        image: python-service:latest
        env:
        - name: BOOTSTRAP_PROFILE
          value: "prod"

Advanced Features

Accessing Nested Configuration

from pyremoteconfig import config

# Dot notation for nested keys
db_host = config("database.host")
redis_port = config("redis.port", cast=int)

Configuration with Vault Secrets

Secrets stored in HashiCorp Vault are automatically fetched:

# In Vault at path: secret/python-service
database:
  password: super-secret-password
api:
  key: api-key-from-vault

# In your code
from pyremoteconfig import config

# Vault secrets are merged with config
db_password = config("database.password")
api_key = config("api.key")

Configuration Refresh

from pyremoteconfig import ConfigClient

client = ConfigClient()

# Force refresh from server
client.refresh()

# Get fresh value
new_value = client.get("feature.flag")

Testing

Mock Configuration for Tests

# tests/conftest.py
import pytest
from unittest.mock import patch

@pytest.fixture
def mock_config():
    test_config = {
        "database.host": "localhost",
        "database.port": "5432",
        "api.key": "test-key",
    }
    
    with patch('pyremoteconfig.config') as mock:
        mock.side_effect = lambda key, default=None, cast=str: \
            cast(test_config.get(key, default))
        yield mock

# tests/test_app.py
def test_database_connection(mock_config):
    from app import get_database_url
    
    url = get_database_url()
    assert "localhost:5432" in url

Troubleshooting

Common Issues

Config server unreachable:

# Add timeout and retry
from pyremoteconfig import ConfigClient

client = ConfigClient(
    timeout=5,
    retry_count=3,
    retry_delay=1
)

Profile not loading:

# Verify environment variable
echo $BOOTSTRAP_PROFILE

# Check bootstrap.yml location
ls -la bootstrap.yml

Vault token expired:

# Refresh Vault token
vault token renew

# Or use app role authentication
export VAULT_ROLE_ID=xxx
export VAULT_SECRET_ID=xxx

Summary