Reverse engineering

Every now and then I find myself reading source code of some of the commercial products we use at my office. I won't specify the products' names, but I will mention some of the biggest vendors are in the list. When it comes to open source software, there is nothing wrong with what I'm doing: I'm having a problem, Google doesn't help much, after some time I decide it'll be best to see how the problematic piece of software is written and after some more time, the issue is solved (and a patch suggestion is submitted). But, when it comes to commercial products, the process involves and extra stage: reverse engineering. Also, almost never a code-fix suggestion is sent to the vendor.
So, what do we reverse engineer at our office? Things that don't require much to "reverse", such as Perl and JSP (yes, vendors do supply these) and things that require decompiling, such as Java classes (usually packed inside archives of some sort). Most major software companies use these technologies, so after reverse engineering one's code, it's not difficult to do this for another. The problem begins when trying to understand how the code works, and where is the culprit. No matter if it's Java of Perl, such code is usually unreadable. Usually, few hours or so are required to make progress.
Here are some examples for reasons for what we do:
* Product installation fails with no obvious reason. This is where the vendor blames our environment, and we have to prove it the problems lies at their code.
* Some product's features are failing. See above.
* The product is badly documented, if documented at all. The only way to understand the way it expects inputs or using resources, is by finding the right place in the code.
* Some programming framework (which are usually open source, such as Struts), is misbehaving, or deeper understanding is required.
One final word about this topic: Every professional developer and a system administrator should be able to do such tricks. Otherwise, one will have to rely on answers found in Google, or worse, patches from the vendors, both of which might come too little and too late.