 |
International Journal for Research Trends and Innovation
International Peer Reviewed & Refereed Journals, Open Access Journal
ISSN Approved Journal No: 2456-3315 | Impact factor: 8.14 | ESTD Year: 2016
Scholarly open access journals, Peer-reviewed, and Refereed Journals, Impact factor 8.14 (Calculate by google scholar and Semantic Scholar | AI-Powered Research Tool) , Multidisciplinary, Monthly, Indexing in all major database & Metadata, Citation Generator, Digital Object Identifier(DOI)
|
Issue: March 2026
Volume 11 | Issue 3
Review Result and Publication of Paper within : 2-3 days
Click Here For more DetailsFor Authors
Forms / Download
Published Issue Details
Editorial Board
Other IMP Links
Facts & Figure
Impact Factor : 8.14
Issue per Year : 12
Volume Published : 11
Issue Published : 118
Article Submitted : 21508
Article Published : 8513
Total Authors : 22399
Total Reviewer : 805
Total Countries : 158
Indexing Partner
Licence
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
|
Published Paper Details
|
|
| Paper Title: | Performance analysis of a Non-Concurrent Crawler, a Concurrent Crawler (Python and Go), and a script using asyncio and aiohttp |
| Authors Name: | P. Manohar Venkat , R. Joshitha , Kannadasan |
| Download E-Certificate: | Download |
| Author Reg. ID: |
IJRTI_182281
|
| Published Paper Id: | IJRTI2206093 |
| Published In: | Volume 7 Issue 6, June-2022 |
| DOI: | |
| Abstract: | Developing a concurrent crawler using Python and Go and comparing its performance with a single-threaded crawler Due to the issues that Python has with concurrency due to GIL (Global Interpreter Lock), the default Python scrapers, and crawlers are relatively slow. The first version will have no concurrency and will just request each website one at a time. The second version makes use of concurrent futures’ thread pool executor, allowing me to send concurrent requests by making use of threads. This paper will solve these issues by using asyncio and aiohttp, which will allow concurrent requests to be made via an event loop. Then see if altering the implementation language has any effect on the speed of retrieval, internal processing, or memory needs for conducting the crawl. Google introduced GO to allow for increased concurrency. |
| Keywords: | Web Crawler, GIL (Global Interpreter Lock), asyncio, aiohttp, Python scrapers. |
| Cite Article: | "Performance analysis of a Non-Concurrent Crawler, a Concurrent Crawler (Python and Go), and a script using asyncio and aiohttp", International Journal for Research Trends and Innovation (www.ijrti.org), ISSN:2455-2631, Vol.7, Issue 6, page no.559 - 566, June-2022, Available :http://www.ijrti.org/papers/IJRTI2206093.pdf |
| Downloads: | 000205401 |
| ISSN: |
2456-3315 | IMPACT FACTOR: 8.14 Calculated By Google Scholar| ESTD YEAR: 2016 An International Scholarly Open Access Journal, Peer-Reviewed, Refereed Journal Impact Factor 8.14 Calculate by Google Scholar and Semantic Scholar | AI-Powered Research Tool, Multidisciplinary, Monthly, Multilanguage Journal Indexing in All Major Database & Metadata, Citation Generator |
| Publication Details: |
Published Paper ID: IJRTI2206093
Registration ID:182281
Published In: Volume 7 Issue 6, June-2022
DOI (Digital Object Identifier):
Page No: 559 - 566 Country: Vellore, Tamil Nadu, India Research Area: Computer Science & Technology Publisher : IJ Publication Published Paper URL : https://www.ijrti.org/viewpaperforall?paper=IJRTI2206093 Published Paper PDF: https://www.ijrti.org/papers/IJRTI2206093 |
| Share Article: | |
|
Click Here to Download This Article |
|
| Article Preview | |
|
|
|
Major Indexing from www.ijrti.org
| Google Scholar | ResearcherID Thomson Reuters | Mendeley : reference manager | Academia.edu |
| arXiv.org : cornell university library | Research Gate | CiteSeerX | DOAJ : Directory of Open Access Journals |
| DRJI | Index Copernicus International | Scribd | DocStoc |
ISSN Details
 |
 |
|
ISSN: 2456-3315
Impact Factor: 8.14 and ISSN APPROVED,
Journal Starting Year (ESTD) : 2016
DOI (A digital object identifier)
| Providing A digital object identifier by DOI.ONE How to Get DOI? |
Conference
Open Access License Policy
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Important Details
Join RMS/Earn 300
WhatsApp
Click Here
Click Here
Indexing Partner |
|||
| Copyright © 2026 - All Rights Reserved - IJRTI | |||