This post covers two topics – how to perform DNS over HTTPS lookups in NET Core, and how to force the HttpClient to bypass its internal DNS resolution process. Why….?

There’s a couple of applications that I’ve written, primarily for home automation, where I’ve needed to intercept traffic from an IoT device – an air conditioner, a weather station etc. This generally meant impersonating the original web service, and redirecting DNS entries to a local server. Generally I’d try to avoid doing that, and rather simply communicate with the same web/cloud service that the IoT device uses, but there have been times when I’ve needed to directly impersonate the cloud service.

This diagram shows the standard flows for an IoT device using a cloud service. The red lines represent DNS lookups, and the blue lines represent application traffic.

The problem of course with impersonating the cloud service, is it means that you’re preventing the IoT device from communicating to the cloud service, and generally breaking the native phone app supplied by the manufacturer. In the diagram below, the IoT device uses local DNS which resolves to the local impersonated cloud service. Meanwhile, the Phone app is still communicating with the cloud service – but not seeing any data from the IoT device.

I needed to find a way of impersonating the cloud service (redirecting DNS), but still allow that web service to communicate directly with the original cloud service at its original IP address. The diagram below shows the final state. The impersonated cloud service is able to use external DNS to find the real address of the cloud service, and then forward data appropriately.

Unfortunately the NET Core HttpClient (System.Net.Http.HttpClient) only uses the underlying host/container’s DNS resolver. That makes sense, generally an application shouldn’t be trying to resolve hosts itself. Now there’re two ways of letting the application bypass the local DNS (with the redirected entries), and instead use an external DNS server (rather than fix it in the infrastructure).

• Import a DNS library/package and generate/send/manage DNS query packets.
• Use DNS over HTTPS (DoH).

This post will focus on DNS over HTTPS (rather than UDP). So that’s one problem fixed. However, knowing the real IP address of the target web server doesn’t completely resolve our issue, you still need to find a way of instructing the HttpClient to use an alternate IP address whilst maintaining the correct Host header.

I fixed the second problem by using the Proxy configuration on the HttpClient. Whilst not quite the traditional use of a proxy configuration on a web request, you can instruct the HttpClient to use the IP address of the real web server as a proxy server, and then make a standard web request. The HttpClient will not perform a local DNS lookup on the host, and instead just ship the web request off directly to the proxy IP address – which happens to be the target web server. The target web server then treats it as a standard HTTP request and sends the expected response.

Here’s the code for the DNS lookup – heavily stripped of logging, caching, etc.

private static HttpClient _httpClient = null;
private static int _iWaitTime = 5000; //ms

static Proxy()
{
	HttpClientHandler httpClientHandler = null;

	httpClientHandler = new HttpClientHandler();
	httpClientHandler.Proxy = null;
	httpClientHandler.UseProxy = false;

	_httpClient = new HttpClient(httpClientHandler);

	_httpClient.DefaultRequestHeaders.Connection.Add("close");
}

public static async Task<IPAddress> GetTargetAddress(string strHost)
{
	HttpResponseMessage httpResponse = null;
	CancellationTokenSource cancellationToken = null;
	IPAddress ipResult = null; 
	dynamic dResults;
			
	try
	{
		cancellationToken = new CancellationTokenSource();
		cancellationToken.CancelAfter(_iWaitTime);

		httpResponse = await _httpClient.GetAsync(string.Format("https://dns.google.com/resolve?name={0}&type=A", strHost), cancellationToken.Token);

		if (httpResponse.IsSuccessStatusCode)
		{
			dResults = JsonConvert.DeserializeObject(await httpResponse.Content.ReadAsStringAsync());

			foreach (dynamic dAnswer in dResults.Answer)
			{
				if (dAnswer.type == "1") // A
				{
					ipResult = IPAddress.Parse(dAnswer.data.ToString());
					break;
				}
			}
		}
		else
			goto Cleanup;
	}
	catch (Exception eException)
	{
		// Error Logging
		goto Cleanup;
	}

Cleanup:
	cancellationToken?.Dispose();
	httpResponse?.Dispose();

	return ipResult;
}

Here’s the code for the proxy.

public static async void ForwardDataToOriginalWebService(string strUserAgent, string strContentType, string strHost, string strPath, string strData)
{
	HttpClient httpClient = null;
	HttpClientHandler httpClientHandler;
	HttpResponseMessage httpResponse = null;
	CancellationTokenSource cancellationToken = null;
	StringContent stringContent;
	IPAddress ipProxy;
	string strContent;
	string strURL = string.Format("http://{0}{1}", strHost, strPath);

	ipProxy = await GetTargetAddress(strHost);
	if (ipProxy == null)
	{
		// Error Logging
		return;
	}

	httpClientHandler = new HttpClientHandler();
	httpClientHandler.Proxy = new WebProxy(string.Format("http://{0}:80", ipProxy.ToString()));
	httpClientHandler.UseProxy = true;

	httpClient = new HttpClient(httpClientHandler);

	httpClient.DefaultRequestHeaders.Connection.Add("close");
	httpClient.DefaultRequestHeaders.UserAgent.ParseAdd(strUserAgent);

	stringContent = new StringContent(strData);
	stringContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(strContentType);

	try
	{
		cancellationToken = new CancellationTokenSource();
		cancellationToken.CancelAfter(_iWaitTime);

		httpResponse = await httpClient.PostAsync(strURL, stringContent, cancellationToken.Token);

		if (httpResponse.IsSuccessStatusCode)
			strContent = await httpResponse.Content.ReadAsStringAsync();
		else
			// Error Logging
	}
	catch (Exception eException)
	{
		// Error Logging
	}

	cancellationToken?.Dispose();
	httpResponse?.Dispose();
	httpClient?.Dispose();
}

You’d generally avoid impersonating a cloud web service, but if you have to do it – this is a good way of enabling both your impersonated service and the original to co-exist. It’s also a good opportunity to experiment with DNS over HTTPS. DNS over HTTPS provides a secure and efficient lookup mechanism for an application to resolve DNS queries. However, 99 times out of 100, the application is going to be relying on the infrastructure for DNS resolution – particularly enterprise applications.

~ Mike