Logging raw HTTP request/response in ASP.NET MVC & IIS7

Definitely use an IHttpModule and implement the BeginRequest and EndRequest events.

All of the "raw" data is present between HttpRequest and HttpResponse, it just isn't in a single raw format. Here are the parts needed to build Fiddler-style dumps (about as close to raw HTTP as it gets):

request.HttpMethod + " " + request.RawUrl + " " + request.ServerVariables["SERVER_PROTOCOL"]
request.Headers // loop through these "key: value"
request.InputStream // make sure to reset the Position after reading or later reads may fail

For the response:

"HTTP/1.1 " + response.Status
response.Headers // loop through these "key: value"

Note that you cannot read the response stream so you have to add a filter to the Output stream and capture a copy.

In your BeginRequest, you will need to add a response filter:

HttpResponse response = HttpContext.Current.Response;
OutputFilterStream filter = new OutputFilterStream(response.Filter);
response.Filter = filter;

Store filter where you can get to it in the EndRequest handler. I suggest in HttpContext.Items. There can then get the full response data in filter.ReadStream().

Then implement OutputFilterStream using the Decorator pattern as a wrapper around a stream:

/// <summary>
/// A stream which keeps an in-memory copy as it passes the bytes through
/// </summary>
public class OutputFilterStream : Stream
{
    private readonly Stream InnerStream;
    private readonly MemoryStream CopyStream;

    public OutputFilterStream(Stream inner)
    {
        this.InnerStream = inner;
        this.CopyStream = new MemoryStream();
    }

    public string ReadStream()
    {
        lock (this.InnerStream)
        {
            if (this.CopyStream.Length <= 0L ||
                !this.CopyStream.CanRead ||
                !this.CopyStream.CanSeek)
            {
                return String.Empty;
            }

            long pos = this.CopyStream.Position;
            this.CopyStream.Position = 0L;
            try
            {
                return new StreamReader(this.CopyStream).ReadToEnd();
            }
            finally
            {
                try
                {
                    this.CopyStream.Position = pos;
                }
                catch { }
            }
        }
    }


    public override bool CanRead
    {
        get { return this.InnerStream.CanRead; }
    }

    public override bool CanSeek
    {
        get { return this.InnerStream.CanSeek; }
    }

    public override bool CanWrite
    {
        get { return this.InnerStream.CanWrite; }
    }

    public override void Flush()
    {
        this.InnerStream.Flush();
    }

    public override long Length
    {
        get { return this.InnerStream.Length; }
    }

    public override long Position
    {
        get { return this.InnerStream.Position; }
        set { this.CopyStream.Position = this.InnerStream.Position = value; }
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        return this.InnerStream.Read(buffer, offset, count);
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        this.CopyStream.Seek(offset, origin);
        return this.InnerStream.Seek(offset, origin);
    }

    public override void SetLength(long value)
    {
        this.CopyStream.SetLength(value);
        this.InnerStream.SetLength(value);
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        this.CopyStream.Write(buffer, offset, count);
        this.InnerStream.Write(buffer, offset, count);
    }
}

---

The following extension method on HttpRequest will create a string that can be pasted into fiddler and replayed.

namespace System.Web
{
    using System.IO;

    /// <summary>
    /// Extension methods for HTTP Request.
    /// <remarks>
    /// See the HTTP 1.1 specification http://www.w3.org/Protocols/rfc2616/rfc2616.html
    /// for details of implementation decisions.
    /// </remarks>
    /// </summary>
    public static class HttpRequestExtensions
    {
        /// <summary>
        /// Dump the raw http request to a string. 
        /// </summary>
        /// <param name="request">The <see cref="HttpRequest"/> that should be dumped.       </param>
        /// <returns>The raw HTTP request.</returns>
        public static string ToRaw(this HttpRequest request)
        {
            StringWriter writer = new StringWriter();

            WriteStartLine(request, writer);
            WriteHeaders(request, writer);
            WriteBody(request, writer);

            return writer.ToString();
        }

        private static void WriteStartLine(HttpRequest request, StringWriter writer)
        {
            const string SPACE = " ";

            writer.Write(request.HttpMethod);
            writer.Write(SPACE + request.Url);
            writer.WriteLine(SPACE + request.ServerVariables["SERVER_PROTOCOL"]);
        }

        private static void WriteHeaders(HttpRequest request, StringWriter writer)
        {
            foreach (string key in request.Headers.AllKeys)
            {
                writer.WriteLine(string.Format("{0}: {1}", key, request.Headers[key]));
            }

            writer.WriteLine();
        }

        private static void WriteBody(HttpRequest request, StringWriter writer)
        {
            StreamReader reader = new StreamReader(request.InputStream);

            try
            {
                string body = reader.ReadToEnd();
                writer.WriteLine(body);
            }
            finally
            {
                reader.BaseStream.Position = 0;
            }
        }
    }
}

---

You can use the ALL_RAW server variable to get the original HTTP headers sent with the request, then you can get the InputStream as usual:

string originalHeader = HttpHandler.Request.ServerVariables["ALL_RAW"];

check out: http://msdn.microsoft.com/en-us/library/ms524602%28VS.90%29.aspx

---

Well, I'm working on a project and did, maybe not too deep, a log using the request params:

Take a look:

public class LogAttribute : ActionFilterAttribute
{
    private void Log(string stageName, RouteData routeData, HttpContextBase httpContext)
    {
        //Use the request and route data objects to grab your data
        string userIP = httpContext.Request.UserHostAddress;
        string userName = httpContext.User.Identity.Name;
        string reqType = httpContext.Request.RequestType;
        string reqData = GetRequestData(httpContext);
        string controller = routeData["controller"];
        string action = routeData["action"];

        //TODO:Save data somewhere
    }

    //Aux method to grab request data
    private string GetRequestData(HttpContextBase context)
    {
        StringBuilder sb = new StringBuilder();

        for (int i = 0; i < context.Request.QueryString.Count; i++)
        {
            sb.AppendFormat("Key={0}, Value={1}<br/>", context.Request.QueryString.Keys[i], context.Request.QueryString[i]);
        }

        for (int i = 0; i < context.Request.Form.Count; i++)
        {
            sb.AppendFormat("Key={0}, Value={1}<br/>", context.Request.Form.Keys[i], context.Request.Form[i]);
        }

        return sb.ToString();
    }

You can decorate your controllers class for log it entirely:

[Log]
public class TermoController : Controller {...}

or log just some individual action methods

[Log]
public ActionResult LoggedAction(){...}

---

Any reason you need to keep it in managed code?

It is worth mentioning that you can enable Failed Trace logging in IIS7 if you don't like re-inventing the wheel. This logs headers, the request and response body as well as many other things.

https://i.stack.imgur.com/XaZ4y.png

---

I went with McKAMEY's approach. Here's a module I wrote that will get you started and hopefully save you some time. You'll need to plug the Logger obviously with something that works for you:

public class CaptureTrafficModule : IHttpModule
{
    public void Init(HttpApplication context)
    {
        context.BeginRequest += new EventHandler(context_BeginRequest);
        context.EndRequest += new EventHandler(context_EndRequest);
    }

    void context_BeginRequest(object sender, EventArgs e)
    {
        HttpApplication app = sender as HttpApplication;

        OutputFilterStream filter = new OutputFilterStream(app.Response.Filter);
        app.Response.Filter = filter;

        StringBuilder request = new StringBuilder();
        request.Append(app.Request.HttpMethod + " " + app.Request.Url);
        request.Append("\n");
        foreach (string key in app.Request.Headers.Keys)
        {
            request.Append(key);
            request.Append(": ");
            request.Append(app.Request.Headers[key]);
            request.Append("\n");
        }
        request.Append("\n");

        byte[] bytes = app.Request.BinaryRead(app.Request.ContentLength);
        if (bytes.Count() > 0)
        {
            request.Append(Encoding.ASCII.GetString(bytes));
        }
        app.Request.InputStream.Position = 0;

        Logger.Debug(request.ToString());
    }

    void context_EndRequest(object sender, EventArgs e)
    {
        HttpApplication app = sender as HttpApplication;
        Logger.Debug(((OutputFilterStream)app.Response.Filter).ReadStream());
    }

    private ILogger _logger;
    public ILogger Logger
    {
        get
        {
            if (_logger == null)
                _logger = new Log4NetLogger();
            return _logger;
        }
    }

    public void Dispose()
    {
        //Does nothing
    }
}

---

OK, so it looks like the answer is "no you can't get the raw data, you have to reconstruct the request/response from the properties of the parsed objects". Oh well, I've done the reconstruction thing.

---

use a IHttpModule:

    namespace Intercepts
{
    class Interceptor : IHttpModule
    {
        private readonly InterceptorEngine engine = new InterceptorEngine();

        #region IHttpModule Members

        void IHttpModule.Dispose()
        {
        }

        void IHttpModule.Init(HttpApplication application)
        {
            application.EndRequest += new EventHandler(engine.Application_EndRequest);
        }
        #endregion
    }
}

    class InterceptorEngine
    {       
        internal void Application_EndRequest(object sender, EventArgs e)
        {
            HttpApplication application = (HttpApplication)sender;

            HttpResponse response = application.Context.Response;
            ProcessResponse(response.OutputStream);
        }

        private void ProcessResponse(Stream stream)
        {
            Log("Hello");
            StreamReader sr = new StreamReader(stream);
            string content = sr.ReadToEnd();
            Log(content);
        }

        private void Log(string line)
        {
            Debugger.Log(0, null, String.Format("{0}\n", line));
        }
    }

---

if for occasional use, to get around a tight corner, how about something crude like below?

Public Function GetRawRequest() As String
    Dim str As String = ""
    Dim path As String = "C:\Temp\REQUEST_STREAM\A.txt"
    System.Web.HttpContext.Current.Request.SaveAs(path, True)
    str = System.IO.File.ReadAllText(path)
    Return str
End Function

---

You can accomplish this in a DelegatingHandler without using the OutputFilter mentioned in other answers in .NET 4.5 using the Stream.CopyToAsync() function.

I'm not sure on the details, but it does not trigger all of the bad things that happen when you attempt to directly read the response stream.

Example:

public class LoggingHandler : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        DoLoggingWithRequest(request);
        var response = await base.SendAsync(request, cancellationToken);
        await DoLoggingWithResponse(response);
        return response;
    }

    private async Task DologgingWithResponse(HttpResponseMessage response) {
        var stream = new MemoryStream();
        await response.Content.CopyToAsync(stream).ConfigureAwait(false);     
        DoLoggingWithResponseContent(Encoding.UTF8.GetString(stream.ToArray()));

        // The rest of this call, the implementation of the above method, 
        // and DoLoggingWithRequest is left as an exercise for the reader.
    }
}

---

I know it's not managed code, but I'm going to suggest an ISAPI filter. It's been a couple of years since I've had the "pleasure" of maintaining my own ISAPI but from what I recall you can get access to all this stuff, both before and after ASP.Net has done it's thing.

http://msdn.microsoft.com/en-us/library/ms524610.aspx

If a HTTPModule isn't good enough for what you need, then I just don't think there is any managed way of doing this in the required amount of detail. It's gonna be a pain to do though.

---

I agree with the others, use an IHttpModule. Take a look at the answer to this question, which does almost the same thing that you are asking. It logs the request and response, but without headers.

How to trace ScriptService WebService requests?

---

It might be best to do this outside of your application. You can set up a reverse proxy to do things like this (and much more). A reverse proxy is basically a web server that sits in your server room, and stands between your web server(s) and the client. See http://en.wikipedia.org/wiki/Reverse_proxy

---

Agree with FigmentEngine, IHttpModule appears to be the way to go.

Look into httpworkerrequest, readentitybody and GetPreloadedEntityBody.

To get the httpworkerrequest you need to do this:

(HttpWorkerRequest)inApp.Context.GetType().GetProperty("WorkerRequest", bindingFlags).GetValue(inApp.Context, null);

where inApp is the httpapplication object.

---

HttpRequest and HttpResponse pre MVC used to have a GetInputStream() and GetOutputStream() that could be used for that purpose. Haven't look into those part in MVC so Im not sure they are availavle but might be an idea :)

---