Apple Safari Browser Developer Options-- a great tool for examining image boards

[deepsepia]

I use a Mac, but maybe this works in the PC version of Safari, or maybe not . . .

Anyway, in Safari, you have the option to enable the "Develop" Menu
(Preferences>Advanced>tick the box marked "show Develop menu in Menu Bar")

Any time you hit a page where you're trying to figure out how a particular thumbnail is finding the fullsize image, enable the "web inspector" from the Develop menu.

This gives the ability to see each element in a page, and where its coming from. You can download any element from the page-- basically you can figure out any Javascript obfuscation. It does not help with Java viewers, or Flash viewers, so far as I can see.

There's a similar tool in Firefox, but the Safari one is much cleaner and easier to use, IMO

[cybormatt]

Well my friend, I'm a well trained web inspector   I think this is a very useful tip for the other users who are not so understanding of the code level stuff.  The hard thing to do is find out how an image is getting there when you're downloading the page on the code level.  Sometimes, javascript and php will add the image after the main page has been sent.  From the code level, this can make finding the image even more difficult.  However, in the main window, it is designed to already process all that stuff... so finding the image is not nearly as difficult.

This feature of Safari that you speak of would be a nice way to find images that are being loaded when javascript is responsible... I can imagine, however, there are some cases where it would stump even Safari.

If you could put some screen caps and a short guide on this, I'd be very interested in seeing this in action.

Matt

[deepsepia]

Well my friend, I'm a well trained web inspector   I think this is a very useful tip for the other users who are not so understanding of the code level stuff.

That would be me. I haven't done any Javascript or PHP in a long time, and when I did it, I did it badly. So I look for the "cheap tricks" to solve things . . .

  The hard thing to do is find out how an image is getting there when you're downloading the page on the code level.  Sometimes, javascript and php will add the image after the main page has been sent.  From the code level, this can make finding the image even more difficult.  However, in the main window, it is designed to already process all that stuff... so finding the image is not nearly as difficult.

I rely on cheap tricks, as I say. The best "cheap trick" is simply the search function: when a big mass of code shows up, I just search for *.jpg . . . its surprising how often that will lead you to the base address of the image

This feature of Safari that you speak of would be a nice way to find images that are being loaded when javascript is responsible... I can imagine, however, there are some cases where it would stump even Safari.

So far, the things that are impossible are various Flash galleries-- they're maddening, because I know there's a full size image there, somewhere, but I can't "find" it. Any tips for examining Flash frames?

I mostly look at the various image sites, and they're pretty straightforward . . . whether its Yahoo Groups or SmugMug, what they've done is stuck a JPG somewhere, and then hidden the address fairly poorly, but its there. I don't have the code skills to systematically find them all . . . that's why IHG is so magical.

If you could put some screen caps and a short guide on this, I'd be very interested in seeing this in action.

Matt

Will do . . . small way of saying "thank you" for your very hard and much appreciated work!

[cybormatt]

I can't say that I've ever really worked with a "flash" gallery of images.  So I can't really help you there.  If you can provide a link to a "flash" gallery, I can see what I can muster up.

As far as smugmug is concerned, it is a perfect example where you need javascript enabled in order to get the image.  Try disabling javascript and then searching for the image.  The javascript uses a function that takes various variable names (about 12), puts them all together, then sends a "POST" request to the server, and then the server sends a JSON file back.  The JSON file contains all the image locations.... then another javascript functions takes the JSON file, parses it, and puts the image locations on the page.  At the code level, I do not evaluate the javascript on the pages... that's just plain unsafe, and hard to do anyway.  But, I do have to figure out how is the javascript loading the image.  For many sites, it's fairly simple to find it in the javascript.  The actual location of the image is embedded in the javascript somewhere. No problem.  Like you say, just search for the *.jpg.  But for smugmug, the location of the image is not embedded anywhere in the javascript.  The javascript makes a second request to the website to find the locations.  What a pain in the ass!!!!

Anyway, that's a long winded explanation!  I would greatly appreciate your post on Safari's wonderful tool   I look forward to it.

matt

[deepsepia]

Anyway, that's a long winded explanation!  I would greatly appreciate your post on Safari's wonderful tool   I look forward to it.

matt

That's an EXCELLENT explanation of something pretty complicated. I have to say, figuring out the tricks that they use is kinda fun . ..  a real puzzle, and obviously one which isn't documented anywhere.

So for that excellent explanation, I surely do owe you an explanation of Safari's inspector tool . . . coming up "real soon now"

[cybormatt]

Sweet!  I look forward to it

[deepsepia]

OK, let's take a look at how you can use Safari to find the location of, or download, a protected file in SM

Let's use this file as a sample

Code:

http://www.smugmug.com/popular/all/1/199474043_aEViN#199474043_aEViN

as its right-click-protected. You can't download it by right-click:



so you'll have to explore and find the base url of the jpg

Hit command-option- I to activate the Developer menu, and select the "Large" version of the photo in the webpage.

Select the "Resources" tab (and sort by size, since the pics will always be largest . . . helps with a busy page)


  
    
You'll see the deobfuscated address of the jpg, right there in the resource tab.

You can save that right click protected file either by dragging it from the resource pane to your desktop, or by downloading from the web address.

Code:

http://photos.smugmug.com/Daily-Album/Every-day-or-at-least-every/092348-copy-2/199474043_aEViN-L-1.jpg

Works on Flickr too

Here's a protected Image

Code:

http://www.flickr.com/photos/jimpatterson/4155136271/

So you open the image in the web inspector--


  

 you'll find that Flickr bundles everything up into a big resource file, but if you search it for JPGs, you'll soon come to the plain address of the base JPG

Code:

http://farm3.static.flickr.com/2556/4155136271_edc55baefb.jpg

This is the very simplest way of using the Web Inspector . . . I owe you more tricks than just this, but its a start. Very handy when there's some complex obfuscation to quickly learn what the base addy is.



You could actually implement this technique inside of any browser that uses Apple's Webkit, I think . . . as the Inspector is part of Webkit

http://trac.webkit.org/wiki/WebInspector

[cybormatt]

Sweetness bro!  I'm sure that your Safari brothers will find this useful! 

[deepsepia]

Sweetness bro!  I'm sure that your Safari brothers will find this useful! 

If I understand it correctly, the webkit toolbox should enable a very powerful IHG -- on a webkit browser (eg Chrome or Safari, but not FF)

The basic idea is the fairly complex problem you've described above

As far as smugmug is concerned, it is a perfect example where you need javascript enabled in order to get the image.  Try disabling javascript and then searching for the image.  The javascript uses a function that takes various variable names (about 12), puts them all together, then sends a "POST" request to the server, and then the server sends a JSON file back.  The JSON file contains all the image locations.... then another javascript functions takes the JSON file, parses it, and puts the image locations on the page.  At the code level, I do not evaluate the javascript on the pages... that's just plain unsafe, and hard to do anyway.  But, I do have to figure out how is the javascript loading the image.  For many sites, it's fairly simple to find it in the javascript.  The actual location of the image is embedded in the javascript somewhere. No problem.  Like you say, just search for the *.jpg.  But for smugmug, the location of the image is not embedded anywhere in the javascript.  The javascript makes a second request to the website to find the locations.  What a pain in the ass!!!!

But what the webkit web inspector code does is to give you access to the "real" location of the image resource, without you having to do all that work . . .

The web inspector also gives you a great handle on the back and forth between the web page and resources . . . my web developer friends swear by it. Firebug does somewhat similar stuff for FF, but its much less intuitive, and importantly for developers, doesn't seem to have an API

I'm not a web coder, but I think you have a pretty neat window into any Image host with the webkit inspector tools.

[cybormatt]

Even with the webkit in Safari, someone still has to tell IHG where the picture is. How is IHG supposed to know what to download? 

[n0aX]

Honestly, I've never thought that Firebug could be "less intuitive" than webkit, I guess I'm getting old..

Firebug is lately the best tool to manipulate/edit/develop/deploy/track both generated and computed interfaces of web pages in js/css/net and html layer. I'd suggest your developer friend to take a look at "FirePHP" and then might think about development abilities and supports by Firebug.

To reveal "super hidden image paths", actually you just need to click on "Net" panel in Firebug,viola..

[deepsepia]

Honestly, I've never thought that Firebug could be "less intuitive" than webkit, I guess I'm getting old..

To reveal "super hidden image paths", actually you just need to click on "Net" panel in Firebug,viola..

Wow . . . thanks :thumbsup:

Even with the webkit in Safari, someone still has to tell IHG where the picture is. How is IHG supposed to know what to download? 

Write code that accesses the value in Firebug (as noted by n0aX above) or Web Inspector.  I think both these applications allow calls -- would mean that you'd have to run the browser in the (slower) Firebug or Web Inspector tracking modes, but if I'm understanding it correctly, then you'd just be pulling the locations of base files out, rather than having to deobfuscate them in your own code.

[cybormatt]

I think even more intuitive about Firebug is the "insepct element" feature.  Just click the "Inspect element" button (surrounded by the red box) and hover the mouse over the image.  If you click the mouse button, it will find the line of code where the image is.

Now this would be useful to me if I ran the javascript on each page, then after the page was processed, look for the image.  However, because I don't process javascript on the pages, I can't make benefit of such a tool.

When the web browser requests the page from smugmug, it receives a responding html file.  That html file then tells the web browser the rest of the files it needs to complete the display.  Some common files are style sheets, javascript, and images.  The first two can be embedded into the html, or be downloaded as a separate file.  Anyhow, if the page requests for javascript files, or contains javascript embedded in the page, the browser will then process the javascript.

This affects IHG because IHG does not process the javascript.  Some of the trickier sites do not put the images into the standard image tags like a normal html file.  Instead, they use javascript to modify the page after it has been received by the user.  They modify the page by inserting the image location into the corresponding image tag where the image should reside.

That's why things like firebug and web inspector won't be any use to IHG.  They are great when I'm building a new entry in the host file, but beyond that, I can't use them in IHG.

[deepsepia]

thanks for the very interesting chat.

I find this whole topic fascinating as a game of cat-and-mouse.

Essentially the site "wants" to work with your browser to deliver a JPG, but also "wants" to deliver that JPG in such a way as its hard to see the base address. Ultimately, any such scheme has holes, but as you've noted, the question of just where the hole is determines the most practical approach.

I'm sort of surprised that none of these hosting sites have implemented a client-side SSL-based viewer, as that would be truly tough to break. If you think of something like Citrix, its not at all easy to pick out elements that its displaying. So long as the image is appearing in the native browser window, its gotta be something that the browser can parse.