PHP cURL how to add the User Agent value OR overcome the Servers blocking cURL requests?
You can add a custom User Agent value to a cURL request in PHP using the CURLOPT_USERAGENT option. This can be done by setting the option to a string containing the desired User Agent value before executing the cURL request.
curl_setopt($ch, CURLOPT_USERAGENT, 'MyCustomUserAgent');
If the server is blocking cURL requests, one way to get around this is to make the request appear as though it is coming from a web browser by setting the User Agent to the value of a common web browser.
Another way is to check if the server allows some other type of request method like file_get_contents or stream_context_create.
<?php
$url = "https://jsonplaceholder.typicode.com/posts/1";
$options = ['http' => ['user_agent' => 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36']];
$context = stream_context_create($options);
$response = file_get_contents($url, false, $context);
if ($response === false) {
echo "Failed to retrieve the contents of the URL.";
} else {
echo $response;
}
?>
It's also possible that the server has implemented a CAPTCHA or other type of anti-bot measure that you will need to bypass in order to make successful requests.