Python: search for a string inside a given url, save to a txt the link it points to -


i'm new python , trying write script looks coupons regarding online courses. flow following:

# read .txt list of courses , create vector containing of them  # loop through vector and, each course, check if there's discount online (e.i udemycoupon.discountsglobal.com)  # if there is, print link pointing

creating vector file easy, else seems give me problems. taking example above website, have create, each course, string needed url. tried search web urllib (and urllib2) couldn't (error 403: forbidden). looked other answers none of them seem work.

could please tell me how write particular part of script (considering example "complete python programming course 2016: code using python 3")?

# have string called "course" containing "complete python programming course 2016: code using python 3"  # substitute spaces inside "course" "+" , # utf-8 symbol code (: should %3a)  # string link = "http://udemycoupon.discountsglobal.com/?s=complete+python+programming+course+2016%3a+code+using+python+3" can created.  # if text "100% off free complete python programming..." # or "98% off complete python programming..." # or "97% off complete python programming..." # or every combination of upper/lower case (i think converting lowercase might convenient) contained in "link" url # save link pointing to variable "coupon_link" # print "coupon_link" new .txt file

this tried:

# -*- coding: utf-8 -*-  import urllib.request  open('courses_list.txt') f:     courseslist = f.readlines() # going modified     courseslistunquoted = courseslist # keep original strings  length = len(courseslist)   # create links http://udemycoupon.discountsglobal.com/ in range(0, length):     courseslist[i] = courseslist[i].replace("\n","")     courseslist[i] = courseslist[i].replace(" ","+").lower()     courseslist[i] = courseslist[i].replace(":","%a3")     courseslist[i] = courseslist[i].replace("#","%23")     courseslist[i] = courseslist[i].replace("!","%21")     courseslist[i] = courseslist[i].replace("/","%2f")  # scrape http://udemycoupon.discountsglobal.com/ addressbeginning = "http://udemycoupon.discountsglobal.com/?s="  in range(0, length):     link = addressbeginning + courseslist[i]     urllib.request.urlopen(link) response:         htmlcode = response.read()     ...

but doesn't seem work. error "no module named request". how can access text on webpage?

thank help.

it might possible website trying access has limited access these resources browsers or specific clients. change user-agent setting them specific browser , send request again.


Comments

Post a Comment

Popular posts from this blog

php - How to add and update images or image url in Volusion using Volusion API -

actually have store importing products store product data insert , update image or image url not insert , update. this xml code name datapro.txt. <products_joined> <productcode>3710_012t</productcode> <vendor_partno>eah5450silent/di/1gd3(lp)</vendor_partno> <productname>test product ta</productname> <hideproduct>n</hideproduct> <stockstatus>20</stockstatus> <lastmodified>1/5/2016 10:25:00 am</lastmodified> <lastmodby>2</lastmodby> <productweight>0.9</productweight> <productprice>100</productprice> <productmanufacturer>asus tek</productmanufacturer> <vendor_price>32.69</vendor_price> <numproductssharingstock>0</numproductssharingstock> <categoryids>107</categoryids> <producturl>http://tebkq.mvlce.servertrust.com/productdetails.asp?productcode=3710_012t</producturl...
Read more

Laravel mail error `Swift_TransportException in StreamBuffer.php line 269: Connection could not be established with host smtp.gmail.com [ #0]` -

i developing website contact-us page guest users can send me email using gmail . anytime try send mail using contact-us form, error swift_transportexception in streambuffer.php line 269: connection not established host smtp.gmail.com [ #0] this mail.php return [ 'driver' => env('mail_driver', 'smtp'), 'host' => env('mail_host', 'smtp.gmail.com'), 'port' => 587, 'from' => ['address' => '', 'name' => ''], 'encryption' => env('mail_encryption', 'tls'), 'username' => env('mail_username'), 'password' => env(mail_password), 'sendmail' => '/usr/sbin/sendmail -bs', 'pretend' => env('mail_pretend', false), ]; this .env mail_driver=smtp mail_host=smtp.gmail.com mail_port=587 mail_username=afecjaped@gmail.com mail_password=********...
Read more

c# SetCompatibleTextRenderingDefault must be called before the first -

i tried search exception couldn't find solution on case i'am using code below invoke .net application : assembly assem = assembly.load(data); methodinfo method = assem.entrypoint; var o = activator.createinstance(method.declaringtype); method.invoke(o, null); the application invoked has form , in entrypoint of application : [stathread] static void main() { application.enablevisualstyles(); application.setcompatibletextrenderingdefault(false); //exception application.run(new form1()); } setcompatibletextrenderingdefault must called before first iwin32window object created in application. edit : assembly = assembly.load(data); methodinfo method = a.gettype().getmethod("start"); var o = activator.createinstance(method.declaringtype); method.invoke(o, null); you should create new method skips initialization...
Read more