Resources

What is a Resource?

A resource represents a sustainability indicator in the system. Each resource tracks a specific metric (like CO2 emissions, temperature, or economic indicators) and maintains metadata about the data being collected. Resources serve as the public-facing entity that client applications interact with, while wrappers handle the behind-the-scenes data collection.

Resource Schema

The resource data model is defined in app/schemas/resource.py:

class ResourceBase(BaseModel):
    wrapper_id: str
    name: str
    type: str

class Resource(ResourceBase):
    id: PyObjectId
    startPeriod: Optional[datetime] = None
    endPeriod: Optional[datetime] = None

Field Descriptions

PyObjectId

required

Unique MongoDB ObjectId for the resource

wrapper_id

string

required

UUID of the associated wrapper that collects data for this resource

name

string

required

Human-readable name of the sustainability indicatorExamples:

“CO2 Emissions - Manufacturing Sector”
“Average Temperature - Lisbon”
“Renewable Energy Percentage”

type

string

required

Classification of the resource. Typically "sustainability_indicator"

startPeriod

datetime

Timestamp of the earliest data point collected. Initially null, updated when first data arrives

endPeriod

datetime

Timestamp of the most recent data point collected. Updated continuously as new data arrives

deleted

boolean

default:"false"

Soft delete flag. Resources are never hard-deleted to maintain data integrity

Resource Lifecycle

1. Creation

Resources are typically created automatically when a wrapper is generated with auto_create_resource: true:

async def create_resource(resource_data: ResourceCreate) -> Optional[dict]:
    wrapper_id = resource_data.wrapper_id
    
    # Validate wrapper exists
    wrapper = await db.generated_wrappers.find_one({"wrapper_id": wrapper_id})
    if not wrapper:
        raise ValueError(f"Wrapper with ID '{wrapper_id}' does not exist")
    
    # Create resource with initial null periods
    resource_dict = deserialize(resource_data.dict())
    resource_dict["deleted"] = False
    resource_dict["startPeriod"] = None
    resource_dict["endPeriod"] = None
    
    result = await db.resources.insert_one(resource_dict)
    return await get_resource_by_id(str(result.inserted_id))

The system validates that the specified wrapper_id exists before creating a resource. This ensures referential integrity between resources and wrappers.

2. Data Collection

Once created, the associated wrapper begins collecting data. As data points flow in:

Wrapper fetches data from its configured source
Data points are published to RabbitMQ queues
startPeriod and endPeriod are automatically updated
Data is routed to the data service for storage

3. Active Use

While active, resources can be:

Queried via the REST API
Updated to modify name or type
Monitored for health and data collection status
Stopped temporarily by stopping the wrapper

4. Deletion

Resources use soft deletion to preserve historical data:

async def delete_resource(resource_id: str) -> Optional[ResourceDelete]:
    object_id = ObjectId(resource_id)
    resource = await db.resources.find_one({"_id": object_id})
    if not resource:
        return None
    
    # Soft delete the resource
    result = await db.resources.update_one(
        {"_id": object_id},
        {"$set": {"deleted": True}},
    )
    
    if result.modified_count > 0:
        wrapper_id = resource.get("wrapper_id")
        if wrapper_id:
            # Stop the wrapper process
            stopped = await wrapper_process_manager.stop_wrapper_process(wrapper_id)
            
            # Update wrapper status
            await db.generated_wrappers.update_one(
                {"wrapper_id": wrapper_id},
                {
                    "$set": {
                        "status": WrapperStatus.STOPPED.value,
                        "updated_at": datetime.utcnow(),
                    },
                    "$push": {"execution_log": log_entry},
                },
            )
            
            # Publish deletion event
            await rabbitmq_client.publish(
                settings.RESOURCE_DELETED_QUEUE,
                json.dumps({"resource_id": resource_id, "wrapper_id": wrapper_id}),
            )

Deleting a resource automatically stops its associated wrapper and publishes a deletion event to notify other services.

Resource-Wrapper Relationship

Resources and wrappers have a one-to-one relationship: Key Points:

Each resource is linked to exactly one wrapper via wrapper_id
The wrapper stores the resource_id for bidirectional navigation
Resources represent the what (indicator being tracked)
Wrappers represent the how (data collection mechanism)

CRUD Operations

Create

resource_data = ResourceCreate(
    wrapper_id="a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    name="Average Temperature - Lisbon",
    type="sustainability_indicator"
)
resource = await create_resource(resource_data)

Read

# Get single resource
resource = await get_resource_by_id(resource_id)

# List all resources with pagination
resources = await get_all_resources(skip=0, limit=10)

Update

update_data = ResourceUpdate(
    wrapper_id=resource.wrapper_id,  # Required but unchanged
    name="Monthly Average Temperature - Lisbon",  # Updated
    type="sustainability_indicator"
)
updated = await update_resource(resource_id, update_data)

Partial updates are supported via ResourcePatch, which allows updating only specific fields without providing all required fields.

Delete

result = await delete_resource(resource_id)
if result and result.deleted:
    print(f"Resource {resource_id} deleted successfully")

Error Handling

The resource service implements comprehensive error handling:

try:
    resource = await get_resource_by_id(resource_id)
except InvalidId as e:
    # Invalid ObjectId format
    logger.error(f"Invalid resource ID format: {resource_id}")
    raise ValueError(f"Invalid resource ID: {resource_id}")
except OperationFailure as e:
    # MongoDB operation failed
    logger.error(f"Database operation failed: {e}")
    raise

Common Error Scenarios

Invalid Resource ID

Error: ValueError: Invalid resource IDCause: Provided ID is not a valid MongoDB ObjectIdSolution: Ensure the ID is a 24-character hexadecimal string

Wrapper Not Found

Error: ValueError: Wrapper with ID 'xyz' does not existCause: Attempting to create a resource for a non-existent wrapperSolution: Create the wrapper first, or verify the wrapper_id is correct

Duplicate Resource

Error: ValueError: Resource with this identifier already existsCause: Database unique constraint violationSolution: Check for existing resources with the same wrapper_id

Querying Resources

The service provides flexible querying capabilities:

# Get all active (non-deleted) resources
resources = await db.resources.find({"deleted": False}).to_list(limit)

# Query by wrapper_id
resource = await db.resources.find_one({
    "wrapper_id": wrapper_id,
    "deleted": False
})

# Query by date range
resources = await db.resources.find({
    "deleted": False,
    "endPeriod": {"$gte": start_date, "$lte": end_date}
}).to_list(limit)

Data Segments

While resources track metadata, actual data points are stored separately in data segments:

class DataPoint(BaseModel):
    x: datetime | float  # Timestamp or numeric value
    y: float             # Measurement value

class TimePoint(BaseModel):
    x: datetime
    y: float

class DataSegment(BaseModel):
    resource_id: PyObjectId
    points: List[TimePoint]
    created_at: datetime

Data segments are managed by a separate data service. Resources only track the time range (startPeriod to endPeriod) of collected data.

Best Practices

Always validate wrapper existence

Before creating a resource, ensure the associated wrapper exists and is properly configured.

Use meaningful names

Resource names should clearly describe what is being measured, including relevant context like location or timeframe.

Monitor period updates

Track startPeriod and endPeriod to ensure data collection is working as expected.

Implement soft deletes

Never hard-delete resources. Use the deleted flag to maintain historical records and data integrity.

Handle errors gracefully

Always catch and handle InvalidId and OperationFailure exceptions when working with resources.

Next Steps

Wrappers

Learn how wrappers collect data for resources

Data Sources

Understand the different data source types

API Reference

View the complete Resources API documentation

Architecture

Explore the overall system architecture

Get Started

Core Concepts

Guides

Deployment

What is a Resource?

Resource Schema

Field Descriptions

Resource Lifecycle

1. Creation

2. Data Collection

3. Active Use

4. Deletion

Resource-Wrapper Relationship

CRUD Operations

Create

Read

Update

Delete

Error Handling

Common Error Scenarios

Querying Resources

Data Segments

Best Practices

Next Steps

Wrappers

Data Sources

API Reference

Architecture

Build docs developers (and LLMs) love

Get Started

Core Concepts

Guides

Deployment

​What is a Resource?

​Resource Schema

​Field Descriptions

​Resource Lifecycle

​1. Creation

​2. Data Collection

​3. Active Use

​4. Deletion

​Resource-Wrapper Relationship

​CRUD Operations

​Create

​Read

​Update

​Delete

​Error Handling

​Common Error Scenarios

​Querying Resources

​Data Segments

​Best Practices

​Next Steps

Wrappers

Data Sources

API Reference

Architecture

Build docs developers (and LLMs) love

What is a Resource?

Resource Schema

Field Descriptions

Resource Lifecycle

1. Creation

2. Data Collection

3. Active Use

4. Deletion

Resource-Wrapper Relationship

CRUD Operations

Create

Read

Update

Delete

Error Handling

Common Error Scenarios

Querying Resources

Data Segments

Best Practices

Next Steps