Member-only story
Web Scraping and NLP for Automated e-Learning Content Creation
If you aim to develop an e-learning platform, the first difficulty you have to overcome will probably be creating the course content. Unfortunately, creating course content is the most challenging part of the development process as it consumes a lot of time and energy. Fortunately, this process can be automated with the help of web scraping and natural language processing.
In this post, I will show you step by step how such automation can be achieved using Scrapy and spaCy. In the first step, I will use the Scrapy to scrape news articles from the web. In the second step, I will process the text data by spaCy to convert the news articles to something directly usable in an English e-learning platform.
The result of the above steps will be a JSON file containing 2000 questions in two categories which are ‘Put the words in correct order’ and ‘Find the missing word’. The content of this JSON file, “simple_sentences.json”, then will be used in my demo web application, lingomoo.
If you want to see how all can be put together in a real-life scenario, check out my lingomoo demo application here. In addition to the demo…
