Available Now !

🚀 Try the Beta Version of the PARTAGE Pipeline!

We’re excited to share the beta version of the PARTAGE pipeline, a fully open-source solution designed to help teachers and institutions easily publish, enrich, and integrate Open Educational Resources (OER) into a semantic knowledge graph.

👉 Access it here: GitLab

Pipeline Beta Overview

How It Works:

  1. OER Submission
    Once deployed, the PARTAGE API allows you to:
    • Upload new OER files or register existing ones (you’ll be asked to provide some basic metadata during this step).
    • Serve and assign a unique URI to the new OER via a web server.
  2. Automatic Content Extraction
    Once an OER is submitted, a message is sent to a RabbitMQ broker. One or more extraction workers, running on separate machines if needed, are notified and will:
    • Retrieve and extract text/content from the resource and saves it.
    • Notify the next stage (wikification).
  3. Wikification & Semantic Enrichment
    The wikification workers analyze the extracted text to:
    • Identify key concepts covered in the resource.
    • Assign relevance scores to each concept.
    • This enriched metadata about the OER is then updated in the database.
  4. RDF Mapping & Knowledge Graph Integration
    The semantics worker fetches metadata in batches. This is done periodically to optimize indexing and avoid overloading while computing multiple indexes.
    • Maps OER metadatd to RDF triples.
    • Adds RDF triples to the CLARA Knowledge Graph (hosted on Virtuoso).

Frequently Asked Questions (FAQ)

  1. Why did you choose RabbitMQ instead of Apache Airflow?
    RabbitMQ offers a simple, robust messaging system that is easy to deploy and understand, especially for lightweight or distributed pipelines. It makes the orchestration of tasks (e.g., extraction, enrichment, RDF conversion) modular and scalable. That said, the pipeline can be adapted to run with Apache Airflow if needed.
  2. Can I run the different workers on different machines?
    Yes! Each stage of the pipeline (extraction, wikification, semantics) can run as an independent worker process on different servers or environments, as long as they are subscribed to the RabbitMQ message broker.
  3. What types of resources can I upload?
    You can upload any educational file (PDF, HTML, PPTX, etc.) that qualifies as an Open Educational Resource (OER) as long as it’s supported by the extraction worker. If you are a programmer you can also add new formats by easily extending the BaseExtractor class of the worker.
  4. How can I monitor the progress of my uploaded OER?
    We are working on a feedback system that will let you track the current processing status of your resource, from ingestion to graph integration.
  5. What happens after I upload a resource?
    Your resource goes through automatic processing: text extraction, concept identification (wikification), and semantic transformations. Finally, it is converted to RDF and added to the CLARA Knowledge Graph for reuse and discovery.

What’s Next?

  • đŸ‡«đŸ‡· French Language Support: Currently, we propose the concepts of an OER only in English – French support is on the way!
  • 🔁 User Feedback Loop: We’re working on a feature to let users track the processing status of their submitted OERs.

Want to contribute or deploy your own instance? Head over to our GitLab repo !