Delete cosmos record with no partition key (Orphan record)

Question

Delete cosmos record with no partition key (Orphan record)

Raghavendra 20

Hi, during development I added a json record to cosmos db through ADF copy activity (source - file, sink - cosmos db) but missed defining the partitionKey inside the json. Now, this record is orphaned. I can query and see that the record exists but I am unable to delete it. I tried options like PartitionKey.None and empty string but fails with error "Message: Entity with the specified id does not exist in the system.". Also tried to set time to live for the container to 5 secs since this is new. The record disappears but when I turn off time to live, it shows up again. How can I get rid of the record?

PS : I tried updating the same record by adding a new entry for partition key but it ended up creating a new record (obvious).

Mahesh Kurva 7,570 Reputation points Microsoft External Staff Moderator

2025-08-14T19:36:49.79+00:00
Hi Raghavendra,

Greetings!!

It looks like you're dealing with an orphan record in Cosmos DB that doesn't have a partition key. Since you added the record without defining the partition key, it's causing issues with deletion.

Here are a few workarounds you can try:

Option 1: Use Azure Cosmos DB Data Migration Tool

You can use the Azure Cosmos DB Data Migration Tool to export all documents except the orphaned one, then recreate the container:

Export all valid documents (those with a partition key).

Delete and recreate the container (this will remove the orphaned record).

Re-import the valid documents.

Option 2: Use Change Feed to Filter and Rebuild

If you have Change Feed enabled:

Use Change Feed to read all valid documents.

Write them to a new container.

Delete the old container.

Option 3: TTL (Time to Live) with a Twist

You mentioned TTL works temporarily. Here's a trick:

Keep TTL enabled and wait until the record disappears.

Export all valid data.

Delete and recreate the container while TTL is still active.

Re-import the valid data.

I hope this information helps. Please do let us know if you have any further queries.
Raghavendra 20 Reputation points

2025-08-14T23:33:27.54+00:00

Mahesh, Since I am not the admin, delete and reimport is not an option. As part of managed identity, I do have option to add or remove a record into that cosmos container. I am using the SDK to delete the record. Any other suggestions?

Accepted answer

0 additional answers

Your answer

Mahesh Kurva 7,570 Reputation points Microsoft External Staff Moderator

2025-08-14T19:36:49.79+00:00

Hi Raghavendra,

Greetings!!

It looks like you're dealing with an orphan record in Cosmos DB that doesn't have a partition key. Since you added the record without defining the partition key, it's causing issues with deletion.

Here are a few workarounds you can try:

Option 1: Use Azure Cosmos DB Data Migration Tool

You can use the Azure Cosmos DB Data Migration Tool to export all documents except the orphaned one, then recreate the container:

Export all valid documents (those with a partition key).

Delete and recreate the container (this will remove the orphaned record).

Re-import the valid documents.

Option 2: Use Change Feed to Filter and Rebuild

If you have Change Feed enabled:

Use Change Feed to read all valid documents.

Write them to a new container.

Delete the old container.

Option 3: TTL (Time to Live) with a Twist

You mentioned TTL works temporarily. Here's a trick:

Keep TTL enabled and wait until the record disappears.

Export all valid data.

Delete and recreate the container while TTL is still active.

Re-import the valid data.

I hope this information helps. Please do let us know if you have any further queries.
Raghavendra 20 Reputation points

2025-08-14T23:33:27.54+00:00

Mahesh, Since I am not the admin, delete and reimport is not an option. As part of managed identity, I do have option to add or remove a record into that cosmos container. I am using the SDK to delete the record. Any other suggestions?

Answer 1

TP 134.2K Volunteer Moderator

Hi @Raghavendra ,

I tested using DeleteItemAsync with id and PartitionKey.None and it successfully deleted item that had no partition key. Can you post sample code of how you are attempting to delete the item?

Instead of using code you could navigate to the Cosmos DB account in Azure portal, click on Data Explorer on left, on right drill down into your Database -- Container -- Items, type filter on top so you only see id you want (e.g. SELECT * FROM c WHERE c.id = "<id you want to delete>") click Apply Filter, select the item, click Delete, confirm.

Please click Accept Answer and upvote if the above was helpful.

Thanks.

-TP

Raghavendra 20

Hi TP, I am using a python script to delete the record. So, there is no DeleteItemAsync method. There are multiple orphan entries. So, using portal is not feasible.
Here is my code snippet


import os

from azure.cosmos import CosmosClient, exceptions, PartitionKey


# Read Azure AD credentials from environment variables
tenant_id = os.getenv("AZURE_TENANT_ID")
client_id = os.getenv("AZURE_CLIENT_ID")
client_secret = os.getenv("AZURE_CLIENT_SECRET")

# Define your Cosmos DB details
database_name = os.getenv("COSMOS_COLLECTION")
container_name = os.getenv("COSMOS_TOOL_CONFIG_CONTAINER")
conn_string = os.getenv("COSMOS_DB_CONNECTION_STRING")
cosmos_master_key = os.getenv("COSMOS_DB_MASTER_KEY")

# Initialize the Cosmos DB client
client = CosmosClient(conn_string, cosmos_master_key)

# Get the database and container
database = client.get_database_client(database_name)
container = database.get_container_client(container_name)

# Define the query
query = "SELECT * FROM c"

try:
    # Execute the query
    items = container.query_items(query, enable_cross_partition_query=True)

    item_count = 0
    # Iterate through the results and delete each item
    for item in items:
        print(f'Deleting item with name: {item["id"]}')
        partition_key = item.get("partitionKey", None)
        if partition_key is None:
            partition_key = None
        container.delete_item(item, partition_key=partition_key)
        item_count += 1
        print(item_count)

    print(f"Found {item_count} records")
    print("All records have been deleted.")

except exceptions.CosmosHttpResponseError as e:
    print(f"An error occurred: {e.message}")

TP 134.2K Volunteer Moderator

Hi @Raghavendra I modified code as below and tested against a container with /partitionKey set as partition key, and it only deleted items that didn't have partitionKey property.

import os

from azure.cosmos import CosmosClient, exceptions, partition_key


# Read Azure AD credentials from environment variables
tenant_id = os.getenv("AZURE_TENANT_ID")
client_id = os.getenv("AZURE_CLIENT_ID")
client_secret = os.getenv("AZURE_CLIENT_SECRET")

# Define your Cosmos DB details
database_name = os.getenv("COSMOS_COLLECTION")
container_name = os.getenv("COSMOS_TOOL_CONFIG_CONTAINER")
conn_string = os.getenv("COSMOS_DB_CONNECTION_STRING")
cosmos_master_key = os.getenv("COSMOS_DB_MASTER_KEY")

# Initialize the Cosmos DB client
client = CosmosClient(conn_string, cosmos_master_key)

# Get the database and container
database = client.get_database_client(database_name)
container = database.get_container_client(container_name)

# Define the query
query = "SELECT * FROM c WHERE NOT IS_DEFINED(c.partitionKey)"

try:
    # Execute the query
    items = container.query_items(query, enable_cross_partition_query=True)

    item_count = 0
    # Iterate through the results and delete each item
    for item in items:
        print(f'Deleting item with name: {item["id"]}')
        container.delete_item(item, partition_key.NonePartitionKeyValue)
        item_count += 1
        print(item_count)

    print(f"Found {item_count} records")
    print("All records have been deleted.")

except exceptions.CosmosHttpResponseError as e:
    print(f"An error occurred: {e.message}")

Raghavendra 20 Reputation points

2025-08-16T01:10:06.5+00:00

It works like a charm. Thank you so much!

Share via

Delete cosmos record with no partition key (Orphan record)

0 additional answers

Your answer