pedrosaclout
/
instance_scrape
0
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

saves instance description, does not close tab just yet

Browse Source
master
Pedro Sá Couto 6 years ago
parent 8239206315
commit e52ec2dc37
3 changed files with 122 additions and 13 deletions
Show Stats Download Patch File Download Diff File

BIN
.DS_Store vendored
View File

Binary file not shown.

119
geckodriver.log
Unescape Escape View File

@ -2532,3 +2532,122 @@ Exiting due to channel error.
Exiting due to channel error. Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown [GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error. Exiting due to channel error.
1552752004174 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.7Y1En8ZRodbl"
1552752004603 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752004603 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752006025 Marionette INFO Listening on port 53320
1552752006108 Marionette WARN TLS certificate errors will be ignored for this session
1552752045670 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.mEDLeLfisXKS"
1552752046087 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752046087 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752047478 Marionette INFO Listening on port 53378
1552752047502 Marionette WARN TLS certificate errors will be ignored for this session
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
[Child 5491, Chrome_ChildThread] WARNING: pipe error: Broken pipe: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 718
Exiting due to channel error.
Exiting due to channel error.
1552752136456 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.oLfs1QRQ7Vlp"
1552752136880 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752136880 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752138283 Marionette INFO Listening on port 53442
1552752138296 Marionette WARN TLS certificate errors will be ignored for this session
console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: NetworkError when attempting to fetch resource..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:188:13
console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: NetworkError when attempting to fetch resource..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:188:13
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
1552752281840 Marionette INFO Stopped listening on port 53320
1552752296463 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.QorJcw4nKTd2"
1552752296866 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752296866 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752298238 Marionette INFO Listening on port 53514
1552752298294 Marionette WARN TLS certificate errors will be ignored for this session
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
1552752354337 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.UDCh3mOdEzns"
1552752354726 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752354726 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752356111 Marionette INFO Listening on port 53574
1552752356175 Marionette WARN TLS certificate errors will be ignored for this session
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
1552752385060 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.IyljQ00gaeiV"
1552752385449 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752385450 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752386828 Marionette INFO Listening on port 53624
1552752386907 Marionette WARN TLS certificate errors will be ignored for this session
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
[Child 5577, Chrome_ChildThread] WARNING: pipe error: Broken pipe: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 718
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
1552752485200 mozrunner::runner INFO Running command: "/Applications/Firefox.app/Contents/MacOS/firefox-bin" "-marionette" "-foreground" "-no-remote" "-profile" "/var/folders/bc/txjl5mmd7cb1ngc8vvs687sc0000gn/T/rust_mozprofile.w4m2Z3zwWrai"
1552752485608 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: resource://pdf.js/
1552752485608 addons.webextension.screenshots@mozilla.org WARN Loading extension 'screenshots@mozilla.org': Reading manifest: Invalid host permission: about:reader*
Can't find symbol 'GetGraphicsResetStatus'.
1552752486982 Marionette INFO Listening on port 53695
1552752487050 Marionette WARN TLS certificate errors will be ignored for this session
console.error: BroadcastService:
receivedBroadcastMessage: handler for
remote-settings/monitor_changes
threw error:
Message: Error: Polling for changes failed: NetworkError when attempting to fetch resource..
Stack:
remoteSettingsFunction/remoteSettings.pollChanges@resource://services-settings/remote-settings.js:188:13
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
[GFX1-]: Receive IPC close with reason=AbnormalShutdown
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.
Exiting due to channel error.

16
selenium_instances.py
Unescape Escape View File

@ -28,27 +28,20 @@ d = 1
i = 0 i = 0
p = -1 p = -1
#NOT WORKING
#results are appended here
#results = []
while True: while True:
driver.find_element_by_css_selector('a.list-group-item:nth-child(%d)'%d).click() driver.find_element_by_css_selector('a.list-group-item:nth-child(%d)'%d).click()
instance_url = driver.find_element_by_xpath('//h5') instance_url = driver.find_element_by_css_selector('#modalInstanceInfoLabel')
description = driver.find_element_by_id('modalInstanceInfo-description') description = driver.find_element_by_id('modalInstanceInfo-description')
print ('Instance:') print ('Instance:')
print(instance_url.text) print(instance_url.text)
# NOT WORKING text_file.write("Instance: "+"\n"+instance_url.text+"\n")
#append instance url to a list
#results.append("Instance: "+"\n"+instance_url.text+"\n")
print ('Description:') print ('Description:')
print(description.text) print(description.text)
#NOT WORKING text_file.write("Description: "+"\n"+description.text+"\n"+"\n")
#append instance description to a list
#results.append("Description: "+"\n"+description.text+"\n"+"\n")
time.sleep(1) time.sleep(1)
@ -95,10 +88,7 @@ while True:
d+=1 d+=1
i+=1 i+=1
text_file.write(str(results))
#pprint(results)
#text_file.write(str(results))
text_file.close() text_file.close()
# close the browser # close the browser
driver.close() driver.close()

Write Preview
Loading…
Cancel
Save