You do not have permission to edit this page, for the following reasons:

This IP address has been blocked from editing Wikipedia.
This does not affect your ability to read Wikipedia pages.
Most people who see this message have done nothing wrong. Some kinds of blocks restrict editing from specific service providers or telecom companies in response to recent abuse or vandalism, and can sometimes affect other users who are unrelated to that abuse. Review the information below for assistance if you do not believe that you have done anything wrong.

The IP address or range 54.163.0.0/16 has been blocked (disabled) by ‪Blablubbs‬ for the following reason(s):

The IP address that you are currently using has been blocked because it is believed to be a web host provider or colocation provider. To prevent abuse, web hosts and colocation providers may be blocked from editing Wikipedia.
You will not be able to edit Wikipedia using a web host or colocation provider because it hides your IP address, much like a proxy or VPN.
We recommend that you attempt to use another connection to edit. For example, if you use a proxy or VPN to connect to the internet, turn it off when editing Wikipedia. If you edit using a mobile connection, try using a Wi-Fi connection, and vice versa. If you are using a corporate internet connection, switch to a different Wi-Fi network. If you have a Wikipedia account, please log in.
If you do not have any other way to edit Wikipedia, you will need to request an IP block exemption.

How to appeal if you are confident that your connection does not use a colocation provider's IP address:
If you are confident that you are not using a web host, you may appeal this block by adding the following text on your talk page: ((unblock|reason=Caught by a colocation web host block but this host or IP is not a web host. My IP address is _______. Place any further information here. ~~~~)). You must fill in the blank with your IP address for this block to be investigated. Your IP address can be determined here. Alternatively, if you wish to keep your IP address private you can use the unblock ticket request system. There are several reasons you might be editing using the IP address of a web host or colocation provider (such as if you are using VPN software or a business network); please use this method of appeal only if you think your IP address is in fact not a web host or colocation provider.

Administrators: The IP block exemption user right should only be applied to allow users to edit using web host in exceptional circumstances, and requests should usually be directed to the functionaries team via email. If you intend to give the IPBE user right, a CheckUser needs to take a look at the account. This can be requested most easily at SPI Quick Checkuser Requests. Unblocking an IP or IP range with this template is highly discouraged without at least contacting the blocking administrator.

This block will expire on 17:28, 10 March 2024.

Even when blocked, you will usually still be able to edit your user talk page, as well as email administrators and other editors.

For information on how to proceed, please read the FAQ for blocked users and the guideline on block appeals. The guide to appealing blocks may also be helpful.

Other useful links: Blocking policy · Help:I have been blocked
Your IP address is in a range that has been blocked on all Wikimedia Foundation wikis. The block was made by ‪Jon Kolbert‬. The reason given is Open proxy/Webhost: See the help page if you are affected .
- Start of block: 18:27, 12 November 2023
- Expiry of block: 18:27, 12 November 2028
Your current IP address is 54.163.179.242. The blocked range is 54.163.0.0/16. Please include all above details in any queries you make. If you believe you were blocked by mistake, you can find additional information and instructions in the No open proxies global policy. Otherwise, to discuss the block please post a request for review on Meta-Wiki. You could also send an email to the stewards VRT queue at stewards@wikimedia.org including all above details.

You can view and copy the source of this page:

((short description|High-performance clustered file system))
((Infobox filesystem
| name = GPFS
| developer = [[IBM]]
| full_name = IBM Spectrum Scale
| introduction_date = ((Start date and age|1998))
| introduction_os = [[AIX]]
| partition_id = 
| directory_struct = 
| file_struct = 
| bad_blocks_struct = 
| max_file_size = 8 EB
| max_files_no = 2<sup>64</sup> per file system
| max_volume_size = 8 YB
| dates_recorded = 
| date_range = 
| date_resolution =
| forks_streams = 
| attributes = 
| file_system_permissions = [[POSIX]]
| compression = 
| encryption = yes
| OS = [[IBM AIX|AIX]], [[Linux]], [[Windows Server]]
))
'''GPFS''' ('''General Parallel File System''', brand name '''IBM Storage Scale''' and previously '''IBM Spectrum Scale''')<ref name="IBM_research_gpfs_page">((cite web
  | title = GPFS (General Parallel File System)
  | publisher = IBM
  | url = https://researcher.draco.res.ibm.com/researcher/view_group.php?id=4840
  | access-date = 2020-04-07
  ))</ref> is high-performance [[clustered file system]] software developed by [[IBM]]. It can be deployed in [[shared-disk]] or [[shared-nothing]] distributed parallel modes, or a combination of these. It is used by many of the world's largest commercial companies, as well as some of the [[supercomputer]]s on the [[TOP500|Top 500 List]].<ref name="schmuck02">((cite conference
  | first = Frank
  | last = Schmuck
  |author2=Roger Haskin
  | title = GPFS: A Shared-Disk File System for Large Computing Clusters
  | book-title = Proceedings of the FAST'02 Conference on File and Storage Technologies
  | pages = 231–244
  | publisher = USENIX
  | date = January 2002
  | location = Monterey, California, US
  | url = http://www.usenix.org/events/fast02/full_papers/schmuck/schmuck.pdf
  | isbn = 1-880446-03-0
  | access-date = 2008-01-18))
</ref>
For example, it is the filesystem of the [[Summit (supercomputer)|Summit]]
<ref name="summit_compute_page">((cite web
  | title = Summit compute systems
  | publisher = Oak Ridge National Laboratory
  | url = https://www.olcf.ornl.gov/olcf-resources/compute-systems/summit/
  | access-date = 2020-04-07
  ))</ref>
at [[Oak Ridge National Laboratory]] which was the #1 fastest supercomputer in the world in the November 2019 TOP500 list of supercomputers.<ref name="Nov 2019 top500 supercomputer list">((cite web
  | title = November 2019 top500 list
  | publisher = top500.org
  | url = https://www.top500.org/list/2019/11/
  | access-date = 2020-04-07
  | archive-date = 2020-01-02
  | archive-url = https://web.archive.org/web/20200102235204/https://www.top500.org/list/2019/11/
  | url-status = dead
  ))</ref> Summit is a 200 [[Petaflops]] system composed of more than 9,000 [[POWER9]] processors and 27,000 NVIDIA [[Volta (microarchitecture)|Volta]] [[GPU]]s. The storage filesystem called Alpine<ref name="summit_faq_page">((cite web
  | title = Summit FAQ
  | publisher = Oak Ridge National Laboratory
  | url = https://www.olcf.ornl.gov/olcf-resources/compute-systems/summit/summit-faqs/
  | access-date = 2020-04-07
  ))</ref> has 250 PB of storage using Spectrum Scale on IBM ESS storage hardware, capable of approximately 2.5TB/s of sequential I/O and 2.2TB/s of random I/O.

Like typical cluster filesystems, GPFS provides concurrent high-speed file access to applications executing on multiple nodes of clusters. It can be used with [[IBM AIX|AIX]] clusters, [[Linux]] clusters,<ref>((cite book|chapter=BPAR: A Bundle-Based Parallel Aggregation Framework for Decoupled I/O Execution|publisher=IEEE|date=Nov 2014|doi=10.1109/DISCS.2014.6|title=2014 International Workshop on Data Intensive Scalable Computing Systems|pages=25–32|last1=Wang|first1=Teng|last2=Vasko|first2=Kevin|last3=Liu|first3=Zhuo|last4=Chen|first4=Hui|last5=Yu|first5=Weikuan|isbn=978-1-4673-6750-9|s2cid=2402391))</ref> on Microsoft [[Windows Server]], or a heterogeneous cluster of AIX, Linux and Windows nodes running on [[x86]], [[Power ISA|Power]] or [[IBM Z]] processor architectures. In addition to providing filesystem storage capabilities, it provides tools for management and administration of the GPFS cluster and allows for shared access to file systems from remote clusters.

==History==
GPFS began as the ''Tiger Shark'' file system, a research project at IBM's [[IBM Research - Almaden|Almaden Research Center]] as early as 1993. Tiger Shark was initially designed to support high throughput multimedia applications. This design turned out to be well suited to scientific computing.<ref name="may00">((cite book
  | last = May
  | first = John M.
  | title = Parallel I/O for High Performance Computing
  | publisher = Morgan Kaufmann
  | year = 2000
  | url = https://books.google.com/books?id=iLj516DOIKkC&q=shark+vesta+gpfs&pg=PA92
  | isbn = 978-1-55860-664-7
  | access-date = 2008-06-18
  | page = 92))</ref>

Another ancestor is IBM's ''Vesta'' filesystem, developed as a research project at IBM's [[Thomas J. Watson Research Center]] between 1992 and 1995.<ref name="corbett93">((Cite book|last1=Corbett|first1=Peter F.|last2=Feitelson|first2=Dror G.|last3=Prost|first3=J.-P.|last4=Baylor|first4=S. J.|title=Proceedings of the 1993 ACM/IEEE conference on Supercomputing - Supercomputing '93 |publisher=ACM/IEEE|year=1993|location=Portland, Oregon, United States|pages=472–481|contribution=Parallel access to files in the Vesta file system|doi=10.1145/169627.169786|isbn=978-0818643408|s2cid=46409100))<!--| access-date = 2008-06-18--></ref> Vesta introduced the concept of file partitioning to accommodate the needs of parallel applications that run on high-performance [[Parallel computing|multicomputers]] with [[parallel I/O]] subsystems. With partitioning, a file is not a sequence of bytes, but rather multiple disjoint sequences that may be accessed in parallel. The partitioning is such that it abstracts away the number and type of I/O nodes hosting the filesystem, and it allows a variety of logically partitioned views of files, regardless of the physical distribution of data within the I/O nodes. The disjoint sequences are arranged to correspond to individual processes of a parallel application, allowing for improved scalability.<ref name="corbett96">((Cite journal
  | first1 = Peter F.
  | last1 = Corbett
  | first2 = Dror G.
  | last2 = Feitelson
  | title = The Vesta parallel file system
  | journal = Transactions on Computer Systems
  | volume = 14
  | issue = 3
  | pages = 225–264
  | url = http://www.cs.umd.edu/class/fall2002/cmsc818s/Readings/vesta-tocs96.pdf
  | date = August 1996
  | doi = 10.1145/233557.233558
  | s2cid = 11975458
  | access-date = 2008-06-18
  | archive-date = 2012-02-12
  | archive-url = https://web.archive.org/web/20120212075707/http://www.cs.umd.edu/class/fall2002/cmsc818s/Readings/vesta-tocs96.pdf
  | url-status = bot: unknown
  ))</ref><ref>((cite journal|author1=Teng Wang|author2=Kevin Vasko|author3=Zhuo Liu|author4=Hui Chen|author5=Weikuan Yu|title=Enhance parallel input/output with cross-bundle aggregation|journal=The International Journal of High Performance Computing Applications|volume=30|issue=2|pages=241–256|date=2016|doi=10.1177/1094342015618017|s2cid=12067366))</ref>

Vesta was commercialized as the PIOFS filesystem around 1994,<ref name="corbett95">((cite journal |last=Corbett |first=P. F. |author2=D. G. Feitelson |author3=J.-P. Prost |author4=G. S. Almasi |author5=S. J. Baylor |author6=A. S. Bolmarcich |author7=Y. Hsu |author8=J. Satran |author9=M. Snir |author10=R. Colao |author11=B. D. Herr |author12=J. Kavaky |author13=T. R. Morgan |author14=A. Zlotek |title=Parallel file systems for the IBM SP computers |journal=IBM Systems Journal |volume=34 |issue=2 |pages=222–248 |year=1995 |url=http://www.research.ibm.com/journal/sj/342/corbett.pdf |access-date=2008-06-18 |doi=10.1147/sj.342.0222 |citeseerx=10.1.1.381.2988 |archive-date=2004-04-19 |archive-url=https://web.archive.org/web/20040419115328/http://www.research.ibm.com/journal/sj/342/corbett.pdf |url-status=bot: unknown ))</ref> and was succeeded by GPFS around 1998.<ref name="barrios99">((cite book
 |first        = Marcelo
 |last         = Barris
 |author2      = Terry Jones
 |author3      = Scott Kinnane
 |author4      = Mathis Landzettel Safran Al-Safran
 |author5      = Jerry Stevens
 |author6      = Christopher Stone
 |author7      = Chris Thomas
 |author8      = Ulf Troppens
 |title        = Sizing and Tuning GPFS
 |publisher    = IBM Redbooks, International Technical Support Organization
 |date         = September 1999
 |url          = https://www.redbooks.ibm.com/redbooks/pdfs/sg245610.pdf
 |no-pp        = true
 |access-date  = 2022-12-06
 |page         = see page 1 (''"GPFS is the successor to the PIOFS file system"'')
 |archive-date = 2010-12-14
 |archive-url  = https://web.archive.org/web/20101214215324/https://www.redbooks.ibm.com/redbooks/pdfs/sg245610.pdf
 |url-status   = bot: unknown
))</ref><ref name="snir01">((cite web
  | last = Snir
  | first = Marc
  | title = Scalable parallel systems: Contributions 1990-2000
  | publisher = HPC seminar, Computer Architecture Department, Universitat Politècnica de Catalunya
  |date=June 2001
  | url = http://research.ac.upc.edu/HPCseminar/SEM0001/snir.pdf
  | access-date = 2008-06-18))
</ref> The main difference between the older and newer filesystems was that GPFS replaced the specialized interface offered by Vesta/PIOFS with the standard [[Unix]] [[API]]: all the features to support high performance parallel I/O were hidden from users and implemented under the hood.<ref name="may00"/><ref name="snir01"/> GPFS also shared many components with the related products IBM Multi-Media Server and IBM Video Charger, which is why many GPFS utilities start with the prefix ''mm''—multi-media.<ref>((cite book | title = General Parallel File System Administration and Programming Reference Version 3.1 | publisher = IBM | date = April 2006 | url = https://www.ira.inaf.it/Computing/tecnica/GPFS/GPFS_31_Admin_Guide.pdf ))</ref>((Rp|xi))

GPFS has been available on IBM's [[AIX]] since 1998, on Linux since 2001, and on Windows Server since 2008.

Today it is used by many of the top 500 supercomputers listed on the Top 500 Supercomputing List. Since inception, it has been successfully deployed for many commercial applications including digital media, grid analytics, and scalable file services.

In 2010, IBM previewed a version of GPFS that included a capability known as GPFS-SNC, where SNC stands for Shared Nothing Cluster. This was officially released with GPFS 3.5 in December 2012, and is now known as FPO
<ref name="fpo-pdf">((cite web
  | title = IBM GPFS FPO (DCS03038-USEN-00)
  | publisher = IBM Corporation
  | year = 2013
  | url = http://public.dhe.ibm.com/common/ssi/ecm/en/dcs03038usen/DCS03038USEN.PDF
  | access-date = 2012-08-12
  ))((Dead link|date=January 2020 |bot=InternetArchiveBot |fix-attempted=yes ))</ref> (File Placement Optimizer). This allows it to use locally attached disks on a cluster of network connected servers rather than requiring dedicated servers with shared disks (e.g. using a SAN). FPO is suitable for workloads with high data locality such as shared nothing database clusters such as SAP HANA and DB2 DPF, and can be used as a [[HDFS]]-compatible filesystem.

==Architecture==
((refimprove|section|date=January 2020))
It is a [[clustered file system]].  It breaks a file into blocks of a configured size, less than 1 megabyte each, which are distributed across multiple cluster nodes.

The system stores data on standard block storage volumes, but includes an internal RAID layer that can virtualize those volumes for redundancy and parallel access much like a RAID block storage system.  It also has the ability to replicate across volumes at the higher file level.

Features of the architecture include
* Distributed metadata, including the directory tree. There is no single "directory controller" or "index server" in charge of the filesystem. 
* Efficient indexing of directory entries for very large directories.
* Distributed locking. This allows for full [[POSIX]] filesystem semantics, including locking for exclusive file access.
* Partition Aware. A failure of the network may partition the filesystem into two or more groups of nodes that can only see the nodes in their group. This can be detected through a heartbeat protocol, and when a partition occurs, the filesystem remains live for the largest partition formed. This offers a graceful degradation of the filesystem — some machines will remain working. 
* Filesystem maintenance can be performed online. Most of the filesystem maintenance chores (adding new disks, rebalancing data across disks) can be performed while the filesystem is live. This maximizes the filesystem availability, and thus the availability of the supercomputer cluster itself.

Other features include high availability, ability to be used in a heterogeneous cluster, disaster recovery, security, [[DMAPI]], [[Hierarchical Storage Management|HSM]] and [[Information Lifecycle Management|ILM]].

=== Compared to Hadoop Distributed File System (HDFS) ===
[[Hadoop]]'s HDFS filesystem, is designed to store similar or greater quantities of data on commodity hardware — that is, datacenters without [[RAID]] disks and a [[storage area network]] (SAN).

* HDFS also breaks files up into blocks, and stores them on different filesystem nodes.
* GPFS has full Posix filesystem semantics.
* GPFS distributes its directory indices and other metadata across the filesystem. Hadoop, in contrast, keeps this on the Primary and Secondary Namenodes, large servers which must store all index information in-RAM.
* GPFS breaks files up into small blocks. Hadoop HDFS likes blocks of ((nowrap|64 MB)) or more, as this reduces the storage requirements of the Namenode. Small blocks or many small files fill up a filesystem's indices fast, so limit the filesystem's size.

==Information lifecycle management==
Storage pools allow for the grouping of disks within a file system. An administrator can create tiers of storage by grouping disks based on performance, locality or reliability characteristics.  For example, one pool could be high-performance [[Fibre Channel]] disks and another more economical SATA storage.

A fileset is a sub-tree of the file system namespace and provides a way to partition the namespace into smaller, more manageable units. Filesets provide an administrative boundary that can be used to set quotas and be specified in a policy to control initial data placement or data migration.  Data in a single fileset can reside in one or more storage pools.  Where the file data resides and how it is migrated is based on a set of rules in a user defined policy.

There are two types of user defined policies: file placement and file management.  File placement policies direct file data as files are created to the appropriate storage pool.  File placement rules are selected by attributes such as file name, the user name or the fileset.  File management policies allow the file's data to be moved or replicated or files to be deleted.  File management policies can be used to move data from one pool to another without changing the file's location in the directory structure. File management policies are determined by file attributes such as last access time, path name or size of the file.

The policy processing engine is scalable and can be run on many nodes at once. This allows management policies to be applied to a single file system with billions of files and complete in a few hours.((citation needed|date=January 2014))

==See also==
((Columns-list|colwidth=20em|
* [[Alluxio]]
* [[ASM Cluster File System]] (ACFS)
* [[BeeGFS]]
* [[GFS2]]
* [[Gluster]]
* [[Google File System]]
* [[List of file systems]]
* [[Lustre (file system)]]
* [[MapR FS]]
* [[Moose File System|MooseFS]]
* [[OCFS2]]
* [[Panasas]] PanFS
* [[QFS]]
* [[IBM Scale-out File Services]] – NAS-grid
* [[Shared disk file system]]
* [[Veritas Cluster Server]]
* [[ZFS]]
))

==References==
((Reflist))



((File systems))

((DEFAULTSORT:GPFS))
[[Category:Shared disk file systems]]
[[Category:Distributed file systems supported by the Linux kernel]]
[[Category:IBM file systems]]
[[Category:IBM storage software|Spectrum Scale]]
[[Category:Distributed file systems]]

Pages transcluded onto the current version of this page (help):

GPFS (edit)
Template:Abbr (view source) (template editor protected)
Template:Abbreviation (view source) (semi-protected)
Template:Ambox (view source) (template editor protected)
Template:Category handler (view source) (protected)
Template:Citation needed (view source) (protected)
Template:Cite book (view source) (protected)
Template:Cite conference (view source) (protected)
Template:Cite journal (view source) (protected)
Template:Cite web (view source) (protected)
Template:Columns-list (view source) (template editor protected)
Template:Dead link (view source) (template editor protected)
Template:Delink (view source) (template editor protected)
Template:Div col (view source) (template editor protected)
Template:Div col/styles.css (view source) (template editor protected)
Template:File systems (edit)
Template:Find sources mainspace (view source) (template editor protected)
Template:Fix (view source) (protected)
Template:Fix/category (view source) (template editor protected)
Template:Hlist/styles.css (view source) (protected)
Template:Infobox (view source) (template editor protected)
Template:Infobox file system (edit)
Template:Infobox filesystem (edit)
Template:Main other (view source) (protected)
Template:More citations needed (view source) (template editor protected)
Template:Navbox (view source) (template editor protected)
Template:Nowrap (view source) (protected)
Template:Pagetype (view source) (protected)
Template:R/superscript (view source) (template editor protected)
Template:R/where (view source) (template editor protected)
Template:Refimprove (view source) (template editor protected)
Template:Reflist (view source) (protected)
Template:Reflist/styles.css (view source) (protected)
Template:Rp (view source) (template editor protected)
Template:SDcat (view source) (template editor protected)
Template:Short description (view source) (protected)
Template:Short description/lowercasecheck (view source) (protected)
Template:Sp (view source) (template editor protected)
Template:Start date and age (view source) (template editor protected)
Template:Template other (view source) (protected)
Template:Years or months ago (view source) (template editor protected)
Template:Yesno (view source) (protected)
Module:Arguments (view source) (protected)
Module:Category handler (view source) (protected)
Module:Category handler/blacklist (view source) (protected)
Module:Category handler/config (view source) (protected)
Module:Category handler/data (view source) (protected)
Module:Category handler/shared (view source) (protected)
Module:Check for unknown parameters (view source) (protected)
Module:Citation/CS1 (view source) (protected)
Module:Citation/CS1/COinS (view source) (protected)
Module:Citation/CS1/Configuration (view source) (protected)
Module:Citation/CS1/Date validation (view source) (protected)
Module:Citation/CS1/Identifiers (view source) (protected)
Module:Citation/CS1/Utilities (view source) (protected)
Module:Citation/CS1/Whitelist (view source) (protected)
Module:Citation/CS1/styles.css (view source) (protected)
Module:DecodeEncode (view source) (template editor protected)
Module:Delink (view source) (template editor protected)
Module:Disambiguation/templates (view source) (protected)
Module:Find sources (view source) (template editor protected)
Module:Find sources/config (view source) (template editor protected)
Module:Find sources/links (view source) (template editor protected)
Module:Find sources/templates/Find sources mainspace (view source) (template editor protected)
Module:GetParameters (view source) (template editor protected)
Module:Infobox (view source) (template editor protected)
Module:Infobox/styles.css (view source) (template editor protected)
Module:Message box (view source) (protected)
Module:Message box/ambox.css (view source) (protected)
Module:Message box/configuration (view source) (protected)
Module:Namespace detect/config (view source) (protected)
Module:Namespace detect/data (view source) (protected)
Module:Navbar (view source) (protected)
Module:Navbar/configuration (view source) (protected)
Module:Navbar/styles.css (view source) (protected)
Module:Navbox (view source) (template editor protected)
Module:Navbox/configuration (view source) (template editor protected)
Module:Navbox/styles.css (view source) (template editor protected)
Module:Pagetype (view source) (protected)
Module:Pagetype/config (view source) (protected)
Module:Pagetype/disambiguation (view source) (template editor protected)
Module:Pagetype/setindex (view source) (template editor protected)
Module:Pagetype/softredirect (view source) (template editor protected)
Module:Plain text (view source) (template editor protected)
Module:SDcat (view source) (template editor protected)
Module:String (view source) (protected)
Module:String2 (view source) (template editor protected)
Module:Template wrapper (view source) (template editor protected)
Module:Unsubst (view source) (protected)
Module:Wikitext Parsing (view source) (template editor protected)
Module:Yesno (view source) (protected)

Return to GPFS.

Retrieved from "https://en.wikipedia.org/wiki/GPFS"