Individual webpages can be quickly added and processed by our system. The content parser automatically removes navigation elements, advertisements, and other distracting elements to focus on the main content. This ensures that only the relevant information is processed and tokenized for your use.
Sitemaps provide an efficient way to process entire websites at once. Simply provide any website URL, and our system will automatically locate and process the sitemap, converting each page into searchable content. This is particularly useful for large websites or documentation that needs to be processed in bulk.
Warning: Processing sitemaps with 2000+ pages may experience delays of 30
minutes or more as well as job failures. This is sually because of the
presence of 404 links in the sitemap or global rate rate limiting being hit.
If your sitemap does not reach “File Ready for Use” status within 30
minutes, please reach out to us.
Our audio processing system handles various types of audio content, from podcasts to meeting recordings. The system can distinguish between different speakers and process their content separately, making it easier to work with multi-speaker recordings. This is particularly valuable for meetings, interviews, or any content where speaker separation is important.
Currently we do not support downloading podcasts. You will need to download the audio locally before adding it to duohub.While we cannot officially recommend specific tools for downloading podcasts, here is a general process that some users have reported success with:
Find your podcast on a streaming platform (e.g., Stitcher, Spotify)
Video content is processed by extracting and analyzing the audio track, making it possible to work with content from various sources. Whether you’re working with YouTube videos, educational content, or recorded meetings, our system can handle videos of any length. The audio is processed separately to ensure accurate content extraction.
Currently we do not support downloading videos from YouTube or Vimeo. You will need to download the video locally before adding it to duohub.We cannot recommend any specific tools for downloading videos from YouTube or Vimeo. However, some users have reported success using:
Our document processing supports multiple file formats and automatically extracts text while maintaining the document’s structure. This makes it easy to work with various types of documents, from simple text files to complex PDFs, ensuring that all your written content is properly processed and indexed.