Retrieving data from AJAX generated website

May I ask if anyone ever tried to retrieve data from website which is created by AJAX in website? What kind of tools do you use?

Thanks in advance~

Do you know or even hear of "Same Origin Policy"?


If you solve this restriction, then it has no difference other than a normal website with RESTful API feeding data

TOP

回覆 2# Jackass_TMxCK

Thanks for the help! I'll do further research regarding this area!

TOP

回覆  Jackass_TMxCK

Thanks for the help! I'll do further research regarding this area!
ronstudio 發表於 1/4/2015 08:18 AM


    I think of something, server side cURL has no limitations on some origin policy

Check out PHP simple proxy

TOP

Thanks, so far I notice most solution people using are loading some kind of browser module in order to simulate the web browsing in order to take the data.  At least in Perl,  there is a module which simulate firefox to achieve this.

This is doable,  but really takes more effort and need furtfurther study

TOP

回覆 1# ronstudio


    curl or wget

TOP

The problem of using simple wget is the content of the page are generated with AJAX javascript. So when I use the simple wget, it gets the html structure but nothing of the content which I need to parse.

TOP

提示: 作者被禁止或刪除 內容自動屏蔽

TOP

回覆 7# ronstudio


    use wget and curl to get the message of ajax. you know ajax is a method of communication right? you can easily find the url of the data source, it is easier than crawling the web page, as the contents are always formatted in json or xml.

TOP

回覆  ronstudio


    use wget and curl to get the message of ajax. you know ajax is a method of co ...
chi251155 發表於 5/4/2015 12:06 AM



    Same Origin Policy....

TOP