本帖最後由 javacomhk 於 2021-5-27 17:45 編輯
NodeJS 加 puppeteer 都得嘅。話哂用JavaScript code 響chrome browser console 都用得番。又或者響chrome browser 搵到的 CSS selector 可以搬過去 nodejs 的script 度用。對於website 改版面或者的class or id name 係非常快可以改番正。
E.g. this code javascript snippet can be used to scrape the title, author and date of this sub-forum. Just fire up a chrome or firefox browser and login to this subforum page https://www.hkepc.com/forum/forumdisplay.php?fid=26 and then paste the code in console (( Cmd + Opt + I in mac or Ctrl + Shift + I in win) and Run. The same code base can be used in nodejs script for automation.- document.querySelectorAll('tr').forEach(function(item) {
- // console code needs to convert to object using Object.values().
- const th = Object.values(item.querySelectorAll('th.subject'))[0];
- if (typeof th !== 'undefined' && th !== null) {
- console.log(th.textContent.trim());
- };
- const cite = Object.values(item.querySelectorAll('td.author > cite'))[0];
- if (typeof cite !== 'undefined' && cite !== null) {
- console.log(cite.textContent.trim());
- };
- const em = Object.values(item.querySelectorAll('td.author > em'))[0];
- if (typeof em !== 'undefined' && em !== null) {
- console.log(em.textContent.trim());
- };
- });
複製代碼- // Goto http://www.aastocks.com/tc/stocks/market/calendar.aspx
- var data = Array.from(
- document.querySelectorAll('.crtRow')
- ).map(
- row => Array.from(row.children).map(node => node.textContent.trim())
- ).map(
- (row) => row[0].length === 0 ? [...row.slice(1)] : row
- ).map(
- (row, idx, arr) => {
- if (row.length === 1) return null;
- const getLastMatch = (idx, arr) =>
- arr[idx].length === 4 ? arr[idx] : getLastMatch(idx - 1, arr);
- const match = getLastMatch(idx, arr);
- const isSameDate = row.length === 3;
- console.log(''.concat(match[0],' ',row[1-isSameDate*1],' ',row[2-isSameDate*1],' ',row[3-isSameDate*1],'\n'));
- return {
- date:match[0],
- stock:row[1-isSameDate*1],
- code:row[1-isSameDate*1].slice(-8).slice(0,5),
- industry:row[2-isSameDate*1],
- period:row[3-isSameDate*1]
- }
- }).filter(Boolean);
- console.log(data);
- // copy to clipboard
- copy(data)
複製代碼 |