Unable to rename a directory named space character using ADLS Gen2 REST API

Question

Unable to rename a directory named space character using ADLS Gen2 REST API

Juan Carlos Fernández Umaña 25

The rename operation fails when the source directory contains a space in its name.

I've been trying to understand why this happens. In my Golang code, I have multiple test cases, and the rename operation consistently fails when the directory name contains spaces. The strange thing is that creating a folder with spaces works perfectly fine, but renaming it results in an error.

I also tried replacing the space with %20 in the URI, but that didn’t work either—instead, it saves the folder name with the %20 symbol rather than interpreting it as a space.

I receive the following error message:

RESPONSE 400: The source URI is invalid.

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake"
	"github.com/Azure/azure-sdk-for-go/sdk/storage/azdatalake/directory"
)

func handleError(err error) {
	if err != nil {
		log.Println(err.Error())
	}
}

func main() {
	// Examples
	// 1. Rename a directory without space to a directory without space (PASSED)
	renameDirectory("oldDir01", "newDir01")

	// 2. Rename a directory without space to a directory with space (PASSED)
	renameDirectory("oldDir02", "new Dir02")

	// 3. Rename a directory with space to a directory without space (FAILED) --> RESPONSE 400: The source URI is invalid.
	// Note: The "old Dir" is created successfully, but the rename fails.
	renameDirectory("old Dir03", "newDir03")

	// 4. Raname a directory with parent directory without space (PASSED)
	renameDirectory("oldDir/subDir04", "oldDir/newSubDir04")

	// 5. Rename a directory with parent directory with space (FAILED) --> RESPONSE 400: The source URI is invalid.
	// Note: The "old Dir/subDir" is created successfully, but the rename fails.
	renameDirectory("old Dir/subDir05", "old Dir/newSubDir05")

	// Final Note:
	// The rename operation fails when the source directory contains a space in its name.
	// The error message is not very descriptive, but it seems that the space in the directory name is causing the issue.
	// The rename operation works fine when the source directory does not contain a space in its name.
	// The directory with space in its name is created successfully, but the rename operation fails.
}

func renameDirectory(oldPath, newPath string) {
	accountName, accountKey := os.Getenv("AZURE_STORAGE_ACCOUNT_NAME"), os.Getenv("AZURE_STORAGE_ACCOUNT_KEY")

	containerName := "123456789"
	// Create a directory client
	u := fmt.Sprintf("https://%s.dfs.core.windows.net/%s/%s", accountName, containerName, oldPath)
	fmt.Println("URL: ", u)

	log.Println("Authenticating with account name and key")
	credential, err := azdatalake.NewSharedKeyCredential(accountName, accountKey)
	handleError(err)

	dirClient, err := directory.NewClientWithSharedKeyCredential(u, credential, nil)
	handleError(err)

	log.Printf("Creating directory %s\n", oldPath)
	_, err = dirClient.Create(context.Background(), nil)
	handleError(err)

	log.Printf("Renaming directory %s to %s\n", oldPath, newPath)
	_, err = dirClient.Rename(context.Background(), newPath, nil)
	handleError(err)

	log.Println("Directory renamed successfully")
}

Can anyone clarify why this happens?

Keshavulu Dasari 4,750 Reputation points Microsoft External Staff Moderator

2025-04-03T18:41:48.3233333+00:00

Hi Juan Carlos Fernández Umaña ,

I Understand the issue you are encountering with renaming directories that contain spaces in their names using the Azure Data Lake Storage Gen2 REST API is likely due to improper encoding of the URI. When dealing with URIs, spaces must be encoded as %20, it seems that the encoding might not be handled correctly in your code.

I Suggest few steps to troubleshoot and potentially resolve the issue:

Make sure that both the source and destination paths are properly encoded. Spaces should be replaced with %20 .

Verify the correct usage of the rename operation in the Azure Data Lake Storage Gen2 REST API documentation. Ensure that all required headers and parameters are correctly set.

Ensure that you are using the latest version of the Azure SDK for Go. There might be bug fixes or improvements related to URI handling.

Add additional logging to capture the exact URIs being used in the rename operation. This can help identify if the issue is with the URI formatting.

URL Encoding: The url.PathEscape function is used to encode the old and new paths. This ensures that spaces and other special characters are properly encoded.

Logging: Additional logging is added to capture the exact URLs being used.

Follow-Up: Run the modified code to see if it resolves the issue, If the issue persists, capture and analyze the error messages for further insights.
Keshavulu Dasari 4,750 Reputation points Microsoft External Staff Moderator

2025-04-04T15:59:38.7566667+00:00

Hi Juan Carlos Fernández Umaña ,

If you have any other questions or are still running into more issues, let me know in the "comments" and I would be happy to help you.
Juan Carlos Fernández Umaña 25 Reputation points

2025-04-04T16:05:23.0333333+00:00

I followed your steps carefully, but it's still not working. I suspect this might be a bug in the Go libraries, as I tested the same scenario in Python and it worked correctly.

Keshavulu Dasari 4,750 Microsoft External Staff Moderator

@ Juan Carlos Fernández Umaña ,

Given that the issue persists despite proper URL encoding and that it works correctly in Python, it might indeed be a bug or limitation in the Go SDK for Azure Data Lake Storage Gen2. few additional steps you can take to further diagnose and potentially work around the issue,

Ensure that you are using the latest version of the Azure SDK for Go. There might have been bug fixes or improvements related to this issue.

As a workaround, you can try using the REST API directly in your Go code instead of the SDK. This will help determine if the issue is with the SDK or the underlying API.


package main

import (
"bytes"
"context"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
"os"
"strings"
)

func handleError(err error) {
if err != nil {
log.Println(err.Error())
}
}

func main() {
// Examples
renameDirectory("oldDir01", "newDir01")
renameDirectory("oldDir02", "new Dir02")
renameDirectory("old Dir03", "newDir03")
renameDirectory("oldDir/subDir04", "oldDir/newSubDir04")
renameDirectory("old Dir/subDir05", "old Dir/newSubDir05")
}

func renameDirectory(oldPath, newPath string) {
accountName, accountKey := os.Getenv("AZURE_STORAGE_ACCOUNT_NAME"), os.Getenv("AZURE_STORAGE_ACCOUNT_KEY")
containerName := "123456789"

// URL encode the paths
encodedOldPath := url.PathEscape(oldPath)
encodedNewPath := url.PathEscape(newPath)

u := fmt.Sprintf("https://%s.dfs.core.windows.net/%s/%s", accountName, containerName, encodedOldPath)
fmt.Println("URL: ", u)

// Create the request URL for renaming
renameURL := fmt.Sprintf("https://%s.dfs.core.windows.net/%s/%s?resource=directory&renameSource=/%s/%s", accountName, containerName, encodedNewPath, containerName, encodedOldPath)
fmt.Println("Rename URL: ", renameURL)

// Create the HTTP request
req, err := http.NewRequest("PUT", renameURL, bytes.NewBuffer(nil))
handleError(err)

// Add headers
req.Header.Add("x-ms-version", "2021-04-10")
req.Header.Add("Authorization", "Bearer "+accountKey)
req.Header.Add("x-ms-rename-source", fmt.Sprintf("/%s/%s", containerName, encodedOldPath))

// Send the request
client := &http.Client{}
resp, err := client.Do(req)
handleError(err)
defer resp.Body.Close()

// Read the response
body, err := ioutil.ReadAll(resp.Body)
handleError(err)

if resp.StatusCode != http.StatusOK {
log.Printf("Failed to rename directory: %s\n", string(body))
} else {
log.Println("Directory renamed successfully")
}
}

This code constructs and sends a direct HTTP request to the Azure Data Lake Storage Gen2 REST API to rename the directory.

The paths are URL-encoded to handle spaces and other special characters, The Authorization header is added with the account key. You might need to adjust this based on your authentication method.

If this approach works, it confirms that the issue is with the Go SDK. If it doesn't, it might indicate a deeper issue with the API or your specific setup.

Accepted answer

0 additional answers

Your answer

Keshavulu Dasari 4,750 Reputation points Microsoft External Staff Moderator

2025-04-03T18:41:48.3233333+00:00

Hi Juan Carlos Fernández Umaña ,

I Understand the issue you are encountering with renaming directories that contain spaces in their names using the Azure Data Lake Storage Gen2 REST API is likely due to improper encoding of the URI. When dealing with URIs, spaces must be encoded as %20, it seems that the encoding might not be handled correctly in your code.

I Suggest few steps to troubleshoot and potentially resolve the issue:

Make sure that both the source and destination paths are properly encoded. Spaces should be replaced with %20 .

Verify the correct usage of the rename operation in the Azure Data Lake Storage Gen2 REST API documentation. Ensure that all required headers and parameters are correctly set.

Ensure that you are using the latest version of the Azure SDK for Go. There might be bug fixes or improvements related to URI handling.

Add additional logging to capture the exact URIs being used in the rename operation. This can help identify if the issue is with the URI formatting.

URL Encoding: The url.PathEscape function is used to encode the old and new paths. This ensures that spaces and other special characters are properly encoded.

Logging: Additional logging is added to capture the exact URLs being used.

Follow-Up: Run the modified code to see if it resolves the issue, If the issue persists, capture and analyze the error messages for further insights.
Keshavulu Dasari 4,750 Reputation points Microsoft External Staff Moderator

2025-04-04T15:59:38.7566667+00:00

Hi Juan Carlos Fernández Umaña ,

If you have any other questions or are still running into more issues, let me know in the "comments" and I would be happy to help you.
Juan Carlos Fernández Umaña 25 Reputation points

2025-04-04T16:05:23.0333333+00:00

I followed your steps carefully, but it's still not working. I suspect this might be a bug in the Go libraries, as I tested the same scenario in Python and it worked correctly.

Answer 1

Hi @Juan Carlos Fernández Umaña

The issue is with the SDK, and you have already raised a GitHub issue for it.

As of now, alternatively, you can use the Azure Data Lake gen2 REST API with a SAS token in your environment.

In my environment, I tried exactly what you did with 5 test cases, creating and renaming directories using the REST API and a SAS token with the Go language.

Code:

package main

import (
	"fmt"
	"log"
	"net/http"
	"net/url"
	"strings"
	"time"
)

var (
	account    = "redacted"
	fileSystem = "redacted"
	sasToken   = "Redacted"
)

func encodePath(path string) string {
	segments := strings.Split(path, "/")
	for i, seg := range segments {
		segments[i] = url.PathEscape(seg)
	}
	return strings.Join(segments, "/")
}

func createDirectory(dir string) {
	encoded := encodePath(dir)
	url := fmt.Sprintf("https://%s.dfs.core.windows.net/%s/%s?resource=directory&%s", account, fileSystem, encoded, sasToken)

	req, err := http.NewRequest("PUT", url, nil)
	if err != nil {
		log.Fatalf("Failed to create request to create directory: %v", err)
	}
	req.Header.Set("x-ms-version", "2021-10-04")
	req.Header.Set("x-ms-date", time.Now().UTC().Format(http.TimeFormat))

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatalf("Create directory request failed: %v", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode >= 200 && resp.StatusCode < 300 {
		fmt.Printf("Created source directory '%s'\n", dir)
	} else {
		fmt.Printf("Failed to create directory '%s': %d %s\n", dir, resp.StatusCode, resp.Status)
	}
}

func renameDirectory(sourceDir, destDir string) {
	sourcePath := encodePath(sourceDir)
	destPath := encodePath(destDir)

	destURL := fmt.Sprintf("https://%s.dfs.core.windows.net/%s/%s?%s", account, fileSystem, destPath, sasToken)
	sourceWithSAS := fmt.Sprintf("/%s/%s?%s", fileSystem, sourcePath, sasToken)

	req, err := http.NewRequest("PUT", destURL, nil)
	if err != nil {
		log.Fatalf("Failed to create rename request: %v", err)
	}
	req.Header.Set("x-ms-version", "2021-10-04")
	req.Header.Set("x-ms-rename-source", sourceWithSAS)
	req.Header.Set("x-ms-date", time.Now().UTC().Format(http.TimeFormat))

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		log.Fatalf("Rename request failed: %v", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode >= 200 && resp.StatusCode < 300 {
		fmt.Printf(" Renamed '%s' → '%s'\n", sourceDir, destDir)
	} else {
		fmt.Printf("Rename failed (%s → %s): %d %s\n", sourceDir, destDir, resp.StatusCode, resp.Status)
	}
}

func main() {
	fmt.Println("---- Starting Rename Tests ----")

	createDirectory("oldDir01")
	renameDirectory("oldDir01", "newDir01")

	createDirectory("oldDir02")
	renameDirectory("oldDir02", "new Dir02")

	createDirectory("old Dir03")
	renameDirectory("old Dir03", "newDir03")

	createDirectory("oldDir/subDir04")
	renameDirectory("oldDir/subDir04", "oldDir/newSubDir04")

	createDirectory("old Dir/subDir05")
	renameDirectory("old Dir/subDir05", "old Dir/newSubDir05")

	fmt.Println("---- All Done ----")
}

Output:

PS C:\Users\xxx\xxxt\goproject\hello> go run hello.go
---- Starting Rename Tests ----
Created source directory 'oldDir01'
 Renamed 'oldDir01' → 'newDir01'
Created source directory 'oldDir02'
 Renamed 'oldDir02' → 'new Dir02'
Created source directory 'old Dir03'
 Renamed 'old Dir03' → 'newDir03'
Created source directory 'oldDir/subDir04'
 Renamed 'oldDir/subDir04' → 'oldDir/newSubDir04'
Created source directory 'old Dir/subDir05'
 Renamed 'old Dir/subDir05' → 'old Dir/newSubDir05'
---- All Done ----

enter image description here

Portal:

enter image description here

Reference:

Path - Create - REST API (Azure Storage Services) | Microsoft Learn

Hope this answer helps! please let us know if you have any further queries. I’m happy to assist you further.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Juan Carlos Fernández Umaña 25 Reputation points

2025-04-07T13:39:58.6666667+00:00

Thank you for this alternative, I will use it for now until the SDK bug is resolved.

Share via

Unable to rename a directory named space character using ADLS Gen2 REST API

0 additional answers

Your answer