python - urllib2.HTTPError: While Web Scraping a huge list -


the web page has huge list of journal names other details. trying scrape table content dataframe. 

#http://www.citefactor.org/journal-impact-factor-list-2015.html  import bs4 bs  import urllib  #using python 2.7 import pandas pd   dfs = pd.read_html('http://www.citefactor.org/journal-impact-factor-list-2015.html/', header=0) df in dfs:     print(df)     df.to_csv('citefactor_list.csv', header=true)

but getting following error .. did try referring raised questions not fix.

error: 

traceback (most recent call last):   file "scrape_impact_factor.py", line 7, in <module>     dfs = pd.read_html('http://www.citefactor.org/journal-impact-factor-list-2015.html/', header=0)   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 896, in read_html     keep_default_na=keep_default_na)   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 733, in _parse     raise_with_traceback(retained)   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 727, in _parse     tables = p.parse_tables()   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 196, in parse_tables     tables = self._parse_tables(self._build_doc(), self.match, self.attrs)   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 450, in _build_doc     return beautifulsoup(self._setup_build_doc(), features='html5lib',   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 443, in _setup_build_doc     raw_text = _read(self.io)   file "/usr/local/lib/python2.7/dist-packages/pandas/io/html.py", line 130, in _read     urlopen(obj) url:   file "/usr/lib/python2.7/contextlib.py", line 17, in __enter__     return self.gen.next()   file "/usr/local/lib/python2.7/dist-packages/pandas/io/common.py", line 60, in urlopen     closing(_urlopen(*args, **kwargs)) f:   file "/usr/lib/python2.7/urllib2.py", line 127, in urlopen     return _opener.open(url, data, timeout)   file "/usr/lib/python2.7/urllib2.py", line 410, in open     response = meth(req, response)   file "/usr/lib/python2.7/urllib2.py", line 523, in http_response     'http', request, response, code, msg, hdrs)   file "/usr/lib/python2.7/urllib2.py", line 448, in error     return self._call_chain(*args)   file "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain     result = func(*args)   file "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default     raise httperror(req.get_full_url(), code, msg, hdrs, fp) urllib2.httperror: http error 500: internal server error

a 500 internal server error means went wrong on server , therefore out of control.

however problem using wrong url.

if go http://www.citefactor.org/journal-impact-factor-list-2015.html/ in browser 404 not found error. remove trailing slash i.e. http://www.citefactor.org/journal-impact-factor-list-2015.html , work.


Comments

Post a Comment

Popular posts from this blog

php - How to add and update images or image url in Volusion using Volusion API -

actually have store importing products store product data insert , update image or image url not insert , update. this xml code name datapro.txt. <products_joined> <productcode>3710_012t</productcode> <vendor_partno>eah5450silent/di/1gd3(lp)</vendor_partno> <productname>test product ta</productname> <hideproduct>n</hideproduct> <stockstatus>20</stockstatus> <lastmodified>1/5/2016 10:25:00 am</lastmodified> <lastmodby>2</lastmodby> <productweight>0.9</productweight> <productprice>100</productprice> <productmanufacturer>asus tek</productmanufacturer> <vendor_price>32.69</vendor_price> <numproductssharingstock>0</numproductssharingstock> <categoryids>107</categoryids> <producturl>http://tebkq.mvlce.servertrust.com/productdetails.asp?productcode=3710_012t</producturl>
Read more

javascript - jQuery UI Splitter/Resizable for unlimited amount of columns -

using jqueryui splitter or resizable functions want create layout allows unlimited amount of columns added , columns need resizable. so fiddle: https://jsfiddle.net/f3glh3mw/ but in fiddle html contains out of nested sets instead of inline. limits possibilities above fiddle can have equal widths 2,4,8,16,32 etc columns. in between shows crooked can see in fiddle. when researching came on question: jquery ui , splitter if @ top answer shows structure like: <div class="wrap"> <div class="resizable resizable1"></div> <div class="resizable resizable2"></div> </div> full fiddle: http://jsfiddle.net/8qztj/86/ which structure want javascript hardcoded 2 resizable items. so here question: how modify last fiddle works amount of .resizable divs. or know of plugin can this. thanks helping! so far found plugin: http://www.bacubacu.com/colresizable/ which says , close functionality want. in
Read more

javascript - IE9 error '$'is not defined -

Image
<!--[if lte ie 9]><html class="lte9"><![endif]--><!--[if gt ie 9]><html><![endif]--> <!--[if !ie]><!--> <html> <!--<![endif]--> blah blah <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.1/jquery.min.js"></script> <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script> <script src="{{ asset('js/mypage.js') }}" type="text/javascript"></script> </body> </html> when load page, ie9 brings error '$' not defined . how can solve error? +++ mypage.js $(function(){ //when load page, //ie9 tag not moving. $(window).load(function() { $('div.mask').hide(); }); // when click , willshow show. //for not ie 9 if (!$('html').hasclass('lte9')) { var = $('ul.about > li > span.activebtn');
Read more