Short story: Why is it like this?

Question

Short story: Why is it like this?

David Arbirk 20

This article reads like an elaborate joke:
invalid-blob-or-block-content-or-invalid-block-list

My story:
Working with azure.. I wanted to upload a large file so I installed the azure cli and tried. Hmm authentication errors. Fix fix, try again error.

Ask chat-gpt ah large files use azcopy. Install azcopy boom, lets go. panic: runtime error: invalid memory address or nil pointer dereference (I have not seen that in a looong time lol)

Chat-gpt: Try sas key (time limited key) ... Error 400 The specified blob or block content is invalid.

start googling.. first article (I didn't believe it):

Yes that is a normal error after failed upload. Your options are:

wait for the garbagecollector to come by in SEVEN days!?!?!?! Wait seven days for the uncommitted blocklist to be cleaned up by garbage collection.
magic

I made a new container and everything worked 🤯

M$ christ wtf get a grip

Nandamuri Pranay Teja 4,855 Reputation points Microsoft External Staff Moderator

2025-08-28T14:23:40.6133333+00:00

Hello David

Thank you for posting the question!

Could you please confirm what authentication hole did you end up going down? The "wrong tenant" maze? Or "your account exists but not in this directory" puzzle? Or the classic "authentication succeeded but you still can't do anything" scenario?

regarding the 7-day wait for cleanup is a known Azure Blob Storage blocks. It’s not a bug but a design choice to allow block uploads to be retried.

For large files you can use try AzCopy: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?tabs=dnf

However, to investigate further please provide the exact error message you received, also confirm What specific version of the Azure CL are you using? Did you have any other configurations, such as network settings or firewalls, that might affect the upload? Did you receive any other error messages apart from the ones mentioned?

Great to hear that creating a new container resolved the issue! however If you plan to keep using the original container, just keep in mind that uncommitted blocks from failed uploads can linger for up to 7 days, potentially causing similar errors until they’re cleaned up automatically.

Let me know if you have questions or concerns via comment.

1 answer

Your answer

Nandamuri Pranay Teja 4,855 Reputation points Microsoft External Staff Moderator

2025-08-28T14:23:40.6133333+00:00

Hello David

Thank you for posting the question!

Could you please confirm what authentication hole did you end up going down? The "wrong tenant" maze? Or "your account exists but not in this directory" puzzle? Or the classic "authentication succeeded but you still can't do anything" scenario?

regarding the 7-day wait for cleanup is a known Azure Blob Storage blocks. It’s not a bug but a design choice to allow block uploads to be retried.

For large files you can use try AzCopy: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10?tabs=dnf

However, to investigate further please provide the exact error message you received, also confirm What specific version of the Azure CL are you using? Did you have any other configurations, such as network settings or firewalls, that might affect the upload? Did you receive any other error messages apart from the ones mentioned?

Great to hear that creating a new container resolved the issue! however If you plan to keep using the original container, just keep in mind that uncommitted blocks from failed uploads can linger for up to 7 days, potentially causing similar errors until they’re cleaned up automatically.

Let me know if you have questions or concerns via comment.

Answer 1

So when I had setup azcopy I got the error below. After that I tried with a temporary "SAS"-key. That worked, but now I had the issue with the blocking un-committed blob.

My main issue in all of this are the unspecific error descriptions. And why are there no indication of issues on the azure portal - surely a warning and a "flush un-committed" button could be made.


C:\Users\<REMOVED>\Downloads\azcopy_windows_amd64_10.30.0\azcopy_windows_amd64_10.30.0> .\azcopy.exe login --tenant-id <REMOVED>
INFO: Authentication is required. To sign in, open the webpage https://microsoft.com/devicelogin and enter the code <REMOVED> to authenticate.
INFO: Login succeeded.
C:\Users\<REMOVED>\Downloads\azcopy_windows_amd64_10.30.0\azcopy_windows_amd64_10.30.0> .\azcopy.exe copy "C:\Users\<REMOVED>\Downloads\xxx.xxx" "https://<REMOVED>.blob.core.windows.net/<REMOVED>" --recursive
INFO: Scanning...
INFO: Autologin not specified.
INFO: Authenticating to destination using Azure AD
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job <REMOVED> has started
Log file is located at: C:\Users\<REMOVED>\.azcopy\<REMOVED>.log

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x10 pc=0x1423ed5]

goroutine 330 [running]:
github.com/Azure/azure-storage-azcopy/v10/ste.(*destReauthPolicy).Do(0xc000090298, 0xc00711a740)
        D:/a/_work/1/s/ste/destReauthPolicy.go:63 +0x1f5
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.(*Request).Next(0xc00711a700)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:146 +0xf0
github.com/Azure/azure-storage-azcopy/v10/ste.statsPolicy.Do({}, 0xc00711a700)
        D:/a/_work/1/s/ste/xferStatsPolicy.go:175 +0x3b
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.(*Request).Next(0xc00710f520)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:146 +0xf0
github.com/Azure/azure-storage-azcopy/v10/ste.logPolicy.Do({{{0xb2d05e00, 0x0}, 0xc00034c750, 0xc00034c768}, 0xc000680780, 0xc0006807b0, 0xc0006807e0}, 0xc00711a6c0)
        D:/a/_work/1/s/ste/xferLogPolicy.go:164 +0x4c5
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.(*Request).Next(0xc00711a680)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:146 +0xf0
github.com/Azure/azure-storage-azcopy/v10/ste.(*retryNotificationPolicy).Do(0xc0014d37e0?, 0xc00711a680)
        D:/a/_work/1/s/ste/xferRetryNotificationPolicy.go:64 +0x2a
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.(*Request).Next(0xc00711a600)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:146 +0xf0
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.telemetryPolicy.Do({{0xc00030c900?, 0x1912a20?}}, 0xc00711a600)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/runtime/policy_telemetry.go:70 +0x1b4
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.(*Request).Next(0xc00711a5c0)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:146 +0xf0
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.includeResponsePolicy(0xc00711a5c0)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/runtime/policy_include_response.go:19 +0x1c
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.PolicyFunc.Do(0x5?, 0x5?)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:216 +0x19
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.(*Request).Next(0xc00711a580)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/request.go:146 +0xf0
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/exported.Pipeline.Do({{0xc00046b980?, 0x0?, 0x0?}}, 0x13a2239?)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.18.1/internal/exported/pipeline.go:76 +0x45
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/internal/generated.(*BlockBlobClient).StageBlock(0xc00034c708, {0x1d1ce10?, 0xc0004cb170?}, {0xc00b12c2a0?, 0x1b003a8ab40?}, 0x1b003a82ef8?, {0x1d1a9e0?, 0xc0006fe360?}, 0xc000060020?, 0x0, ...)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/storage/azblob@v1.6.2/internal/generated/zz_blockblob_client.go:554 +0xcb
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob/blockblob.(*Client).StageBlock(0xc00034c720, {0x1d1ce10, 0xc0004cb170}, {0xc00b12c2a0, 0x30}, {0x1d1a9e0, 0xc0006fe360}, 0xc0014d3d38)
        C:/Users/<REMOVED>/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/storage/azblob@v1.6.2/blockblob/client.go:221 +0x275
github.com/Azure/azure-storage-azcopy/v10/ste.(*blockBlobUploader).GenerateUploadFunc.(*blockBlobUploader).generatePutBlock.func2()
        D:/a/_work/1/s/ste/sender-blockBlobFromLocal.go:104 +0x31a
github.com/Azure/azure-storage-azcopy/v10/ste.(*blockBlobUploader).GenerateUploadFunc.(*blockBlobUploader).generatePutBlock.createSendToRemoteChunkFunc.createChunkFunc.func4(0x5f5e100?)
        D:/a/_work/1/s/ste/sender.go:207 +0x294
github.com/Azure/azure-storage-azcopy/v10/ste.(*jobMgr).chunkProcessor(0xc0005c7008, 0xa3)
        D:/a/_work/1/s/ste/mgr-JobMgr.go:1055 +0xab
created by github.com/Azure/azure-storage-azcopy/v10/ste.(*jobMgr).poolSizer in goroutine 59
        D:/a/_work/1/s/ste/mgr-JobMgr.go:957 +0x236

Nandamuri Pranay Teja 4,855 Reputation points Microsoft External Staff Moderator

2025-08-29T09:59:40.3933333+00:00

Hello David Arbirk

Ensure C:\Users<REMOVED>\Downloads\xxx.xxx is a valid file or directory. If it’s a directory, confirm it contains files, as AzCopy skips empty folders (as noted in the message: "Any empty folders will not be processed").

Run dir C:\Users\<REMOVED>\Downloads\xxx.xxx to verify the path.

Post which Confirm the destination URL (https://<REMOVED>.blob.core.windows.net/<REMOVED>) is correct and accessible. Ensure the storage account name and container name are accurate. Verify you have write permissions to the container using the Azure Portal.

az storage blob list --account-name <storage-account-name> --container-name <container-name> --auth-mode login

The crash occurs in the destReauthPolicy, which handles re-authentication during the transfer. The authentication may have expired or failed during the operation so Re-run the login command: .\azcopy.exe login --tenant-id <REMOVED>.

Let me know if you have any question!

Share via

Short story: Why is it like this?

1 answer

Your answer