You want to use EditUrl when you are navigating to a sub control of your current control (using a Control Key). If you are navigating anywhere else, use NavigateUrl. The main difference, (aside from the different signatures) is that EditUrl will include mid=(your module's ID) in the querystring so that DNN knows how to load the control you are trying to load based on the control key.
The most basic scenario for EditUrl is where you just need to get to a sub control with a control key of "Edit". That's where you'll just use EditUrl() without any parameters. Say that the control key is "DisplayOptions," then use EditUrl("DisplayOptions").
Now, the "tricky" part is when you want to pass custom parameters. For the first parameter, use those KeyName, and KeyValue parameters (so, EditUrl("Action", "Add", "DisplayOptions") ). If you have other parameters, use the AdditionalParamters array, like so: EditUrl("Action", "Add", "DisplayOptions", "pid=1", "lid=2", "xid=3"). The reason this seems kind of weird is that EditUrl is written with the edit scenario in mind, where you'd do something like EditUrl("itemId", 3, "EditItem"), where you often need to keep track of just one parameter (the item you are editing).
All of these parameters should show up on the querystring (all EditUrl is doing, after all, is creating a URL). In the receiving control, you should be able to get at them by calling Request.QueryString("Action"), etc.
If that isn't working, take a look at the URL that EditUrl is generating, and see whether those values are getting put on it.
Hope that helps,