Annual MEDLINE®/PubMed® Year-End Processing (YEP):
NLM® performs both daily routine maintenance and annual year-end processing of MEDLINE data. Throughout the year, revised MEDLINE citations are released daily from the NLM Data Creation and Maintenance System (DCMS). In the fall of each year NLM performs additional maintenance to MEDLINE data, known as YEP, or year-end processing.
Routine Maintenance includes, but is not limited to, the following types of edits:
- corrections of misspelled author names, collaborator names, or title and abstract words;
- the addition of published errata, retractions, and other updates as described in the NLM Fact Sheet, Errata, Retractions, Partial Retractions, Corrected and Republished Articles, Duplicate Publications, Comments (including Author Replies), Updates, Patient Summaries, and Republished (Reprinted) Articles Policy for MEDLINE®;
- the addition of International Standard Serial Numbers (ISSN), or changes to Journal Title Abbreviations and Full Journal Titles;
- changes to the preferred Names for Supplementary Concepts that include chemicals, protocols, and diseases (concepts that are not MeSH descriptors); and
- adjustments to the MeSH® (Medical Subject Headings) or Publication Types (PT) data.
These corrections result from the NLM Quality Assurance reviews as well as from user reports. These maintained citations, along with new citations, are then available to MEDLINE licensees and PubMed users daily.
YEP (year-end processing) encompasses more extensive changes to the MEDLINE citations. This maintenance is primarily for making changes to the Medical Subject Headings (MeSH) on existing MEDLINE citations to conform to the forthcoming year's version of MeSH, and also involves other global changes such as those to Supplementary Concept Names.
The annual year-end processing procedures usually start in mid-November and are targeted for completion by mid-December.
During this year-end processing time, the schedule for adding citations newly indexed with MeSH and revised citations to MEDLINE/PubMed is temporarily interrupted, and no newly indexed or maintained versions of MEDLINE citations are added to PubMed. New in process citations, however, continue to be added.
When the year-end processing activities are complete, the new version of MEDLINE, with the new year's MeSH Headings, is loaded into PubMed, and the regular MEDLINE/PubMed update schedule resumes. At this point PubMed uses the new year's MeSH vocabulary in the MeSH translation tables and the MeSH database.
Entry terms are synonymous or alternate forms of a concept and can be used interchangeably with preferred terms in PubMed.
Use PubMed's Details tab to see how terms are mapped.
Recurring Annual Changes
- Brand New Concepts
- Replaced-by heading; old heading retained as entry term to new heading
- Replaced-by heading; old heading deleted and not retained as entry term to new heading; adjustments to search strategies needed
- Deleted Heading with Follow-On Adjustments
- Deleted Heading; old heading is retained as an entry term for the new heading
- Deleted heading; old heading becomes entry term to new heading but new heading not equivalent in meaning to old headings
- Entry Terms (See References) changes
- Other MeSH changes that are made during year-end processing
- Publication Types
- Identification of Clinical Trials in MEDLINE
- Entry Combination Revisions
- Heading Mapped-To (HM) Maintenance
- EC/RN Numbers (RN) and Name of Substance (NM)
- Supplementary Concept Records (SCRs) Elevated to MeSH Headings
- Pharmacological Action (PA) Added to Supplementary Concept Records
The following types of changes take place during year-end processing:
- The annual update to the Medical Subject Headings (MeSH®), the NLM controlled vocabulary used for subject indexing and retrieval, is implemented in a new Entrez MeSH Database and a new MeSH Browser.
- The annual update to the MEDLINE citations to reflect the annual changes in MeSH vocabulary (new, revised, and deleted terms); this includes both the MeSH Heading (MH) element and the Name of Substance (NM) element and the Enzyme Commission/CAS Registry Number (RN) element.
- Other changes to MEDLINE citations may involve:
Identification of Clinical Trials in MEDLINE
Entry Combination Revisions
Heading Mapped-To (HM) Maintenance
Additional changes and corrections to MEDLINE data are made as deemed necessary for each particular year.
These annual changes and corrections are documented in XML format. See MEDLINE Citation Maintenance section of the MeSH Section homepage.
Examples of recurring annual MeSH changes during year-end processing:
Brand New Concepts
- Each year NLM adds new MeSH terminology to cover new concepts for the MeSH vocabulary. Generally, NLM does not retrospectively re-index MEDLINE citations with new MeSH Headings. Therefore, searching in PubMed for a new MeSH Term qualified as [MeSH Term] or [Major MeSH Topic] effectively limits retrieval to citations indexed after the term was introduced. PubMed's Automatic Term Mapping (ATM) expands an untagged subject search to include both MeSH Term and Text Word(s), and may retrieve relevant citations indexed before the introduction of a new MeSH term.
A new MeSH term, Abdominal Fat, was introduced for 2006 MeSH. A PubMed query on October 3, 2005 for abdominal fat without any search tags yielded 5,774 citations, of which 5,573 contained MeSH Headings. None of these 5,573 will have the new MeSH heading added. Searchers may wish to include headings previously used to index these concepts in addition to the text words PubMed may use to supplement a strategy. For Abdominal Fat, the previous indexing was Adipose Tissue (1966-2005) as shown in the NLM MeSH Browser for the Abdominal Fat record.
- Exceptions to the NLM policy of using headings only after their introduction include: 1) the addition of MeSH Headings to OLDMEDLINE citations, and 2) adjustments to existing citations when a previously ambiguous heading is redefined (see the example for Seals under the section Deleted Heading with Follow-on Adjustments), and 3) the Heading Mapped-To maintenance.
Replaced-by heading; old heading retained as entry term to new heading
For 2003 MeSH the old MeSH Heading Pineal Body was changed to Pineal Gland. All occurrences of the MeSH Heading Pineal Body on MEDLINE citations indexed from 1966-2002 were changed to Pineal Gland. Pineal Body was retained as an entry term so that searches will retrieve the same MeSH-indexed citations whether using the new preferred form of Pineal Gland or the older form of Pineal Body.
For 2004 MeSH, the old MeSH Heading Progestational Hormones was changed to Progestins. The MeSH Heading occurrences of Progestational Hormones on MEDLINE citations indexed from 1966-2003 were all changed to Progestins.
In both examples above, the old version of the heading was retained as an entry term to the new heading, so no adjustments to searching were necessary, unless the searcher wanted the text word portion (as provided by the PubMed Automatic Term Mapping) to use the more up-to-date vocabulary.
Replaced-by heading; old heading deleted and not retained as entry term to new heading; adjustments to search strategies needed
For 2004 MeSH the MeSH heading Anabolic Steroids was deleted. All occurrences of that term in MEDLINE citations were replaced by the new MeSH heading Anabolic Agents. In this case, the old version of the heading was not retained as an entry term to the new heading, so adjustments to searching and stored searches were needed.
For 2003 the MeSH heading Actinobacteria Group was deleted and all occurrences of that term in MEDLINE were replaced by the new MeSH heading Actinobacteria. The old heading, Actinobacteria Group, was not retained as an entry term so beginning with the 2003 version of PubMed, that term no longer yields retrieval.
In 2008 the MeSH Heading "Embryo" was replaced by the MeSH Heading "Embryo, Mammalian." "Embryo" was retained as an Entry Term in MeSH, but it now maps to Embryonic Structures which is a parent term to both Embryo, Mammalian and Embryo, Nonmammalian. This mapping will ensure a comprehensive explosion in PubMed.
NLM performed additional maintenance on existing citations for three scenarios:
Replacing a MeSH Heading with a different MeSH Heading does not change the major status or the subheadings applied. However, if a MeSH Heading is added to a citation it is added without subheadings. The major emphasis (starred heading) is applied.
In 2008 the MeSH Heading "Gene Products, vpu" is replaced by the Supplementary Concept Record, "vpu protein, Human immunodeficiency virus 1." In other words, search this "old" MeSH Heading as: vpu protein, Human immunodeficiency virus 1 [nm]
Deleted Heading with Follow-On Adjustments
In 2003 the MeSH Heading Shrimp was deleted and all occurrences of that term were replaced by the MeSH heading Decapoda (Crustacea). Additional work took place to identify those citations previously indexed with Shrimp that should not have Decapoda (Crustacea) but should have that term replaced with one of the following new more specific MeSH headings as appropriate: Palaemonidae, Pandalidae, or Penaeidae which are all specific families of shrimp. The string "Shrimp" no longer occurs in MeSH because the word is ambiguous; the old single concept was replaced by four more clearly defined concepts. Since 2003 MeSH, any PubMed searches containing the term "Shrimp" no longer map to a MeSH heading.
Another example of retrospectively adjusting citations for new concepts occurred in 2005 when the MeSH heading Seals was deleted and all occurrences of that term were replaced by the MeSH heading Seals, Earless. Additional maintenance work took place to identify those citations previously indexed with Seals that should not be indexed with Seals, Earless. Rather, Seals, Earless should have been replaced with one of the following new more specific MeSH headings as appropriate: Fur Seals or Sea Lions, which are also members of the suborder Pinnipedia along with Seals, Earless and Walruses. Like "Shrimp," the string "Seals" no longer occurs in MeSH because the word is ambiguous and searches containing the term "Seals" no longer map to a MeSH heading.
Deleted Heading; old heading is retained as an entry term for the new heading
The MeSH Heading Delivery was deleted as a MeSH heading in 2003 but not deleted from the 2003 MeSH vocabulary because it got a new status as entry term for the new MeSH heading Delivery, Obstetric. Maintenance tasks were performed to replace the MeSH heading Delivery with the new MeSH heading Delivery, Obstetric but the old term may still be used in searching PubMed for the concept.
Deleted heading; old heading becomes entry term to new heading but new heading not equivalent in meaning to old headings
The mapping of "old" headings and/or see references (entry terms) can also change.
Blacks was a 2003 MeSH Heading that became an entry term to the new 2004 heading African Continental Ancestry Group, but all occurrences of Blacks in MEDLINE citations were changed to African Americans (because the majority of the citations were to the U.S. slant). As a result, the search term Blacks will still retrieve postings, but those hits are not necessarily the exact equivalent of what was retrieved before. Beginning in 2004, the old concept of Blacks is represented by the preferred term of African Americans, which is treed under the African Continental Ancestry Group. Some additional maintenance was performed as well; citations that were originally indexed to Blacks that were also indexed to countries other than the United States had the new MeSH heading African Continental Ancestry Group added too.
From 2003 MeSH here is a summary list of MeSH headings that were replaced by another heading yet the 2003 string maps to yet another heading for 2004, signaling a change in meaning that may affect search results.
|2003 Heading||2004 Replaced-by-Heading||2003 Heading is a See Reference or a Redefined 2004 Heading|
|Animals||Animal Population Groups||Animals|
|Blacks||African Americans||African Ancestry Continental Group|
|Cytochrome a||Cytochrome a Group||Cytochromes a|
|Mannosidosis||alpha-Mannosidosis||Mannosidase Deficiency Diseases|
|Pneumocystis carinii||Pneumocystis||Pneumocystis carinii [redefined MH]|
Entry Terms (See References) changes
Remember that the mapping of entry terms can also change (independently of a preferred string change).
One year, "Allspice" was removed as an Entry Term to Rosales and added as an Entry Term to the MeSH Heading Pimenta. This affects a user's search since a search for Allspice prior to 2002 mapped to Rosales but after 2002 the search maps to a different heading, Pimenta, which is a different concept. The History Note (HN) in the MeSH Browser traces the change and alerts searchers to use different MeSH Headings for different year spans.
Other MeSH changes that are made during year-end processing include:
- On occasion, when NLM finds it necessary, changes may be made to Publication Types (PT). Usually these changes have involved changing a MeSH Heading to a Publication Type; or it may involve deletion of publication types that then may become entry terms to another less specific publication type.
In 2004 and 2005 the more specific Review publication types (Review, Academic, Review, Literature, Review, Multicase, Review, Tutorial and Review of Reported Cases) were deleted and became entry terms to the publication type Review.
Identification of Clinical Trials in MEDLINE
NLM continues its work with the Cochrane Collaboration to enhance the identification of clinical trials in MEDLINE. This work involves the addition of "Randomized Controlled Trial" or "Controlled Clinical Trial" Publication Types to MEDLINE citations identified by the Cochrane Collaboration. In the past this work has mostly been done at year-end-processing but now this work may take place at any time during the year.
Entry Combination Revisions
Each year NLM replaces certain MeSH heading/subheading combinations on MEDLINE records with the appropriate precoordinated MeSH heading or other MeSH heading/subheading combination, known as Entry Combinations.
The combination Lymphatic Vessels/growth & development was changed to Lymphangiogenesis and the combination Bone Marrow Cells/transplantation was changed to Bone Marrow Transplantation.
- Searchers who get zero retrieval on a MeSH heading/subheading combination should check the MeSH Browser for that heading to see if that subheading is listed as an Entry Combination and what the appropriate equivalent is for searching. Here is what the MeSH Browser looks like for the Bone Marrow Cells example:
|MeSH Heading||Bone Marrow Cells|
|Entry Combination||transplantation:Bone Marrow Transplantation|
- Note: It is important to note that when using the Entrez MeSH Database the list of subheadings for a term includes those paired at least once with the heading and may not reflect current rules for allowable combinations. The MeSH Browser has the currently allowable combinations.
Heading Mapped-To (HM) Maintenance
Every SCR (Supplementary Concept Record) Name of Substance (NM) in the MeSH vocabulary is associated with a MeSH Heading and that heading is automatically added to every MEDLINE citation indexed with that NM. The resulting MeSH heading addition is referred to as the "Heading Mapped-to (HM)", although it will display on the citation as a regular MeSH heading (MH). The MeSH Browser shows the current Heading Mapped-to value for every SCR. Heading Mapped-to maintenance adds the current HM value to journal citations as an MH when that MH is not already present on the MEDLINE citations.
When the SCR ESCO2 protein, human is added to a citation, the MeSH Headings, Acetyltransferases and Chromosomal Proteins, Non-Histone will be added to the citation.
|Name of Substance||ESCO2 protein, human|
|Heading Mapped to||*Acetyltransferases|
|Heading Mapped to||*Chromosomal Proteins, Non-Histone|
For more information on Heading Mapped-to maintenance, see the article, Heading Mapped-to Maintenance: for Supplementary Concept Records' Names of Substance.
EC/RN Numbers (RN) and Name of Substance (NM)
Changes in MeSH also cover Enzyme Commission (EC) Nomenclature and CAS Registry Number (RN) data, including Substance Name (NM). These affect not only the MeSH chemical concepts in Category D, but the Supplementary Concept Records as well. The corresponding changes are made to MEDLINE citations during year-end processing.
Supplementary Concept Records (SCRs) Elevated to MeSH Headings
For general information on SCRs elevated to MeSH heading status, see the article, Changes in the Treatment of Chemical Data in MEDLINE® Citations.
Pharmacological Action (PA) Added to Supplementary Concept Records:
As a reminder, a policy, effective August 2002, calls for adding PAs to new or existing SCRs only if the following three criteria are met:
- there must be more than 20 citations discussing the Pharmacological Action being exhibited by that drug;
- there must be substantial evidence that the Pharmacological Action is in effect in humans (i.e., the drug is used clinically); and
- a reasonable proportion of the literature (10%) on that drug accounts for those effects.
The article, Pharmacologic Action Headings: PubMed®, describes changes to PubMed when a Pharmacologic Action term is available.
If you enter an unqualified term that is both a MeSH Term and a Pharmacologic Action Term, PubMed will search the term as [MH], [PA], or [TW].
platelet aggregation inhibitors will automatically map to:
The PA is edited only in the MeSH authority data and used in PubMed Automatic Term Mapping (ATM). Indexers add the PA to MEDLINE citations as part of the indexing process; there is no routine PA maintenance on MEDLINE citations.
Annual NLM Technical Bulletin Articles
- MEDLINE®/PubMed® Year-End Processing (YEP) Activities - September-October
- MEDLINE® Data Changes - November-December
- What's New for MeSH - November-December
- PubMed® Notes - November-December
- Articles in the November-December issue that address changes to LocatorPlus and NLM Catalog, which may be different from MEDLINE.
References to NLM Technical Bulletin Articles and Quick Tours
- Three minute animated Quick Tour on Changing Saved Searches in My NCBI
- Heading Mapped-to Maintenance: for Supplementary Concept Records' Names of Substance
- Changes in the Treatment of Chemical Data in MEDLINE® Citations
- PubMed® and the Expansion of Pharmacological Action Terms
- OLDMEDLINE MeSH® Mapping Project - Update
OLDMEDLINE citations now have MeSH Headings and undergo Year-End Processing. These older citations thus may have a MeSH Heading on them prior to the introduction of the term.