02/11/2021
Hey everyone, we have a new remote position available:
Web Crawling and Data Engineering
We are looking for a talented, passionate, and pragmatic engineer, able to work in a rapidly changing environment. You will work on large-scale web crawlers to extract structured/unstructured data from multiple online sources. If you love playing with data and code, have an eye for detail, and have a strong client delivery mindset, we’d love to hear from you!
what we offer:
-A competitive salary.
-Responsibility from day one in a fast-growing and global startup.
-A vibrant international team.
Job requirements:
key responsibilities.
-Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrape, parse and store.
-Develop frameworks for automating and maintaining the constant flow of data from multiple sources.
-Look to solve problems with an optimistic approach and take on challenges.
-Implement best practices and patterns in design, development, and deployment.
-Creating, using, and maintaining web scraping libraries so that future scrapers can be built faster.
-Work independently with little supervision to do research and test innovative solutions skills.
skills required:
Core skills and experience we are looking for:
-2-6 years of experience in NodeJS, Python with frameworks like -Puppeteer, Selenium, and others.
-Experience with data parsing - knowledge of Regular expression, HTML, CSS, DOM, Javascript, and XPATH.
-Experience with network protocol analyzer like WireShark, Charles Proxy.
-Experience with storing structured & unstructured data in Postgres and AWS S3.
-Strong knowledge of performance optimization, multi-processing, multi - threading and concurrency.
-Experience with AWS cloud services.
-Good knowledge in web scraping and APIs.
-Fluent English
If you wish to apply then please send your resume to the following email: [email protected]