I've installed a module to do a search through our database to display a list of members based on a set of criteria. But when I try to do the basic of basic lines of code to test out the IsPostBack, it doesn't work the way i'm expecting it to (display "rawr" when it's a post back).

The code is not anywhere remotely complicated to display such

If IsPostback Then
   Response.Write("rawr")
End if

I haven't worked with VB.NET in about 2-3 years, so it could be I'm not writing the code right...

The most common place to look for the PostBack status is in the Page_Load event.  There you would do something like:

If Me.Page.IsPostback Then
    ' your code here
End If

I prefer to prefix the "IsPostback" with the referenced object since it is possible to have an object other than the page initiate the postback.

I cannot say that I experience that behavior.  Unless there are other controls or logic changing or bypassing the page life cycle...  This is where you need to look.