uclient-fetch: Use HEAD for --spider

In GNU wget the --spider first issues a HEAD request, then if HEAD fails, issues a GET request.
In uclient, only a GET request is sent.
The patch changes GET to HEAD e.g. get the file size without downloading it first.

This is still not totally compatible with GNU wget because it does not retry with GET if HEAD fails.
Someone may use the --spider to call a GET only API, so they may be affected.
But this is incorrect usage while others may expect that the spider uses HEAD and don't expect a download.

Signed-off-by: Sungbo Eo <mans0n@gorani.run>
Signed-off-by: Sergey Ponomarev <stokito@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>

diff --git a/uclient-fetch.c b/uclient-fetch.c
index 8f4a28220dd46a670e4644f60dc435bb2973cadb..26269f52df076d9b9c75eb5d8d0c3a5ba33d9319 100644 (file)
--- a/uclient-fetch.c
+++ b/uclient-fetch.c
@@ -47,6 +47,7 @@ struct header {
 };
 
 static const char *user_agent = "uclient-fetch";
+static const char *method = NULL;
 static const char *post_data;
 static const char *post_file;
 static char opt_post = 0;  /* 1 when --post-data/file is used */
@@ -345,7 +346,7 @@ static int init_request(struct uclient *cl)
 
        msg_connecting(cl);
 
-       rc = uclient_http_set_request_type(cl, opt_post ? "POST" : "GET");
+       rc = uclient_http_set_request_type(cl, method);
        if (rc)
                return rc;
 
@@ -795,6 +796,15 @@ int main(int argc, char **argv)
                }
        }
 
+       if (opt_post == 1) {
+               method = "POST";
+       } else if (no_output) {
+               /* Note: GNU wget --spider sends a HEAD and if it failed repeats with a GET */
+               method = "HEAD";
+       } else {
+               method = "GET";
+       }
+
        argv += optind;
        argc -= optind;