STS Scanner Tutorial and Other Stuff
January 20th, 2008 by Jordan Del-Grande (Dedicated Page)I have just completed a tutorial on how to use the STS Scanner correctly. I believe it to be comprehensive with detailed commands and screen shots but your comments would be appreciated…You can download the pdf tutorial here.
On a side note, I found an interesting blog that mentioned the Security Technology Science site. I think it is worth mentioning that this site has some really detailed and interesting blogs. For starters, here is a link to their 10 day vulnerability assessment found here. You should also check out their blog on Security Testing Tools of 2007 found here.
Anyway, one thing I would like to clarify is a past article I wrote about screen scraping. I mentioned XPath to be “complex” when in fact I should have used different wording. I should have said that I find the use of XPath when crawling an unknown site to be “complex” in comparison to the solution below. What I mean by this is that using XPath to screen scrape a page that is known to me is indeed straight forward (as long as you understand the XPath syntax and the DOM structure), but what happens when you or your tool have never seen the page before? How do I know there is an anchor tag at html/body/div[4]/div/div[2]/ul/li/a/ and not at html/body/div[4]/div/div[2]/ul/a/? Anyway, maybe there is an easy answer to this and an algorithm has already been written but at the time it just seemed a lot easier in Ruby to get all the links on the one page with one line of code…
page.links.each do { |link| ... }
Finally, the STS Scanner uses Mechanize and Hpricot to parse and spider html pages. These libraries are not able to parse and idenitify JavaScript and/or Ajax code. A separate version of the STS Scanner is planned to be developed where the internal core does not rely on these libraries and hence they shall become obsolete. The core shall act more like an everyday browser (i.e., IE, Firefox, etc) so it can hopefully branch out into other applications such as Flash and web services. So those who thought I was going to solve the Web 2.0 problem with these libraries are mistaken…