The General Index is a free-to-use database, which when compressed takes up 8.5 terabytes. It was created by technologist Carl Malamud and his nonprofit foundation Public Resource. As of 2021, it contains words and phrases from more than 107 million academic papers.[1][2]

It consists of a table of n-grams (a contiguous sequence of n items) derived from the full text of the articles along with tables of associated keywords and metadata.[3] It is intended to ease computerized analysis of the scientific literature, which has been hindered by widespread copyright restrictions limiting access by researchers to the full text.

The initial version, comprising the raw database tables without any search engine front-end, was released by the Internet Archive on October 7, 2021.[1]

See also

References

  1. ^ a b Else, Holly (2021-10-26). "Giant, free index to world's research papers released online". Nature. doi:10.1038/d41586-021-02895-8. PMID 34703019. S2CID 240000069.
  2. ^ ""The General Index": New tool allows you to search 107 million research papers for free". Big Think. Retrieved 2021-11-12.
  3. ^ By (2021-11-03). "Malamud's General Index: Research Gist, No Slap On The Wrist". Hackaday. Retrieved 2021-11-12.

External links

Official website

Categories