Azure Data Factory - No "body" option for GET method into copy data's REST source

Question

Azure Data Factory - No "body" option for GET method into copy data's REST source

Gigi Giorgobiani 5

I have a list of parameters for the REST API call. The method I am trying to call API is Copy Data with REST source, method is GET and there is no body section there.
So where do I exactly specify these parameters? Header is clearly not meant for this purpose. In case of POST calls I could successfully specify the parameters in the body section but there is not body option for GET. User's image

User's image1

QuantumCache 20,356 Reputation points

2023-06-22T17:27:26.14+00:00

Hello @Gigi Giorgobiani, thanks for reaching out on this forum!

Did you try testing your GET call from POSTMAN client?

Is the API an external endpoint?

In the case of a GET request, you can specify the parameters in the URL itself!!!
Gigi Giorgobiani 5 Reputation points

2023-06-22T17:35:58.01+00:00

hello @SatishBoddu-MSFT , yes, it's already tested it by postman and it works.

source API only supports this kind of communication.

to use GET method and body for parameters like:

{

"Content-Type: "application/json",

"BeginDate": "2023-01-01",

"EndDate": "2023-12-31"

}"
Gigi Giorgobiani 5 Reputation points

2023-06-22T21:29:42.5+00:00

@SatishBoddu-MSFT main question is that how is possible to mimic or create GET request with body parameters
QuantumCache 20,356 Reputation points

2023-06-28T20:30:27.44+00:00

Hello @Gigi Giorgobiani,

Unfortunately, Azure Data Factory's built-in HTTP connector does not support sending a GET request with a request body. The connector follows the standard HTTP protocol, which only allows parameters to be sent in the query string of the URL for GET requests.
Ian Martin 10 Reputation points

2023-09-04T15:28:49.4066667+00:00

@Santish;

This is no longer correct.

RFC9110 (the latest definitions of the http standards) differentiates methods purely by the cache and impact of the request (relevant documentation is here: https://www.rfc-editor.org/rfc/rfc9110.html#name-get).

While it is uncommon to have body content on a get, its not unheard of.

The inverse issue of this (a post requirement with no body) is however a common occurance (in fact is it used by a number of microsofts own apis, which are equally hard to hit from azure data factory.

Could we please have bodies optional for all verb types as per the web standards.
QuantumCache 20,356 Reputation points

2023-09-04T16:52:37.0833333+00:00

Hello @Ian Martin Thanks for responding on this case. Let me work with the product team to know more on this before i comment on this topic.
Gordon Little (UK) 0 Reputation points

2024-02-16T11:09:40.44+00:00

I've run into this also. While not recommended, the HTTP Spec does allow for a body being passed with a GET request. The vendor product I'm integrating with has, in their wisdom, implemented a GET method and passes the parameters in the request body. I'm unable to access from data factory.
Lucy Susan Martin 0 Reputation points

2024-02-16T11:19:34.1833333+00:00

@Gordon Little (UK) unfortunate we actually had to run a very stupid little micro-service in K8s that accepts posts, and forwards them as gets to a hardcoded address. It was trivial to set up, but only because we already had a massive cluster. We also had to do similar for a product that required no body post requests - which interestingly enough was a part of SharePoint. By all means, default to good standards, but we need support for contentfull gets and contentless posts. Heck, there are now standards for custom domain specific verbs, surely this is not hard to add in as an "advanced" dropdown.

1 answer

Your answer

QuantumCache 20,356 Reputation points

2023-06-22T17:27:26.14+00:00

Hello @Gigi Giorgobiani, thanks for reaching out on this forum!

Did you try testing your GET call from POSTMAN client?

Is the API an external endpoint?

In the case of a GET request, you can specify the parameters in the URL itself!!!
Gigi Giorgobiani 5 Reputation points

2023-06-22T17:35:58.01+00:00

hello @SatishBoddu-MSFT , yes, it's already tested it by postman and it works.

source API only supports this kind of communication.

to use GET method and body for parameters like:

{

"Content-Type: "application/json",

"BeginDate": "2023-01-01",

"EndDate": "2023-12-31"

}"
Gigi Giorgobiani 5 Reputation points

2023-06-22T21:29:42.5+00:00

@SatishBoddu-MSFT main question is that how is possible to mimic or create GET request with body parameters
QuantumCache 20,356 Reputation points

2023-06-28T20:30:27.44+00:00

Hello @Gigi Giorgobiani,

Unfortunately, Azure Data Factory's built-in HTTP connector does not support sending a GET request with a request body. The connector follows the standard HTTP protocol, which only allows parameters to be sent in the query string of the URL for GET requests.
Ian Martin 10 Reputation points

2023-09-04T15:28:49.4066667+00:00

@Santish;

This is no longer correct.

RFC9110 (the latest definitions of the http standards) differentiates methods purely by the cache and impact of the request (relevant documentation is here: https://www.rfc-editor.org/rfc/rfc9110.html#name-get).

While it is uncommon to have body content on a get, its not unheard of.

The inverse issue of this (a post requirement with no body) is however a common occurance (in fact is it used by a number of microsofts own apis, which are equally hard to hit from azure data factory.

Could we please have bodies optional for all verb types as per the web standards.
QuantumCache 20,356 Reputation points

2023-09-04T16:52:37.0833333+00:00

Hello @Ian Martin Thanks for responding on this case. Let me work with the product team to know more on this before i comment on this topic.
Gordon Little (UK) 0 Reputation points

2024-02-16T11:09:40.44+00:00

I've run into this also. While not recommended, the HTTP Spec does allow for a body being passed with a GET request. The vendor product I'm integrating with has, in their wisdom, implemented a GET method and passes the parameters in the request body. I'm unable to access from data factory.
Lucy Susan Martin 0 Reputation points

2024-02-16T11:19:34.1833333+00:00

@Gordon Little (UK) unfortunate we actually had to run a very stupid little micro-service in K8s that accepts posts, and forwards them as gets to a hardcoded address. It was trivial to set up, but only because we already had a massive cluster. We also had to do similar for a product that required no body post requests - which interestingly enough was a part of SharePoint. By all means, default to good standards, but we need support for contentfull gets and contentless posts. Heck, there are now standards for custom domain specific verbs, surely this is not hard to add in as an "advanced" dropdown.

Answer 1

JASON WELCH 0

For anyone else who arrives here with the same issue <sigh>:

Change verb to POST
Enter your required Body content in the box
Save
Change verb back to GET - the Body UI will disappear
Save and run

Although the Body content is no longer visible in the UI, a quick debug will show that the previously-entered Body is sent with the GET verb.

Thanks Microsoft - a bug that actually does something useful :D

Share via

Azure Data Factory - No "body" option for GET method into copy data's REST source

1 answer

Your answer