In the world of web reconnaissance and bug hunting, we've seen useful tools such as  LinkFinder, relative-url-extractor, and JSParser. They all do what they promise: find links.

But last year, as I was looking to improve my own bug-hunting tooling, I looked at what was currently out there to add another layer of recon/enumeration data. All these solutions couldn’t quite achieve what I was hoping to accomplish. That lead me down the path of creating jswzl, standing on the shoulders of these giants. Below are some of the insights that started everything.

1. Reliance on Regular Expressions (Regex)

The subject of regex is not something that really needs a lot of reiteration, as others have written at length about the woes of using regular expressions. All the solutions that exist have used Regex, which means that it struggles to understand things like this:

I tried to solve examples like this with regex also, but quickly realized that it was never going to work. A proper set of Abstract Syntax Tree (AST) walkers, and a light runtime where needed, ends up being the key to being able to solve some of the problems outlined below.

2. Limited scope

Links and paths are really great things to find. But when you dig into JavaScript, you discover that JavaScript files contain so much more information, helping you save time. Consider things like:

● What data does a request include?

● What headers are set on requests?

● What controllers/views/routes does the front¬-end have?

These are all things that can be real timesavers to have in front of you when assessing an application. And they are all relatively trivial to extract!

3. Incompatibility with packed, minified, or chunked sources

You can have the most advanced AST walker or set of regexes, but they will be inherently blind if they don’t take into account sources that are packed, minified, and chunked:

● If it’s minified, code will often be mangled to the point where it’s unreadable in the name of making it take up fewer bytes.

● If it’s chunked, code is regularly only lazy loaded when needed. This means you need to either pre-fetch chunks or spider the application at runtime to hope to load everything.

● If it’s packed, you can often get much higher fidelity code out of it with a bit of processing.

As the scope of these tools is inherently limited by design, they leave a lot of information on the table. That’s a shame!

4. Lack of understanding of source maps

Source maps are wonderful. They give you close to the original source code, pre-packing/minification/chunking. Sometimes, they are inline, and sometimes, they exist as .js.map files.

But because these tools don’t integrate with tools like Burp Suite, they can’t fetch source maps where relevant, and they don’t apply any source map that even exists in-line.

5. Lack of context in output

These tools will generally output HTML reports, or plain-text output in list form. That’s great. But when you start to actually test an application in-depth, you often want far more context to understand what a path is for and what it’s trying to accomplish.

As they don’t show the code context, they only serve as a starting point and you end up needing another tool to view the full source code anyway.

6. Being side projects

At first, jswzl was a side project too. But as I started to realize the value that a toolchain focused on solving these issues could bring, I understood that there was a significant need for somebody to tackle these problems considering the long-term outlook.

But make no mistake: The road was paved by these open-source projects, and they deserve immense credit for the value they have brought and continue to bring!

Given how essential JavaScript is to the web and for understanding how modern Single-Page Applications (SPA) work, it became clear that somebody had to have a go at creating these tools so that the larger community can leverage its capabilities and improve the efficacy of testing.

Great minds think alike

Just recently, TomNomNom over at BishopFox released the tool jsluice. This is the first publicly released tool that I’ve found that makes use of an AST to tackle some of these problems.

I was delighted to see somebody as smart as Tom realize that this was a problem that deserved to be solved.

Where do we go from here?

Stay tuned for more information about how jswzl tackles these problems. I’ll be sharing a lot more information in the coming days about how jswzl works and how it can help you understand and better test JavaScript-heavy web applications!

‍