You do not have permission to edit this page, for the following reasons:

This IP address has been blocked from editing Wikipedia.
This does not affect your ability to read Wikipedia pages.
Most people who see this message have done nothing wrong. Some kinds of blocks restrict editing from specific service providers or telecom companies in response to recent abuse or vandalism, and can sometimes affect other users who are unrelated to that abuse. Review the information below for assistance if you do not believe that you have done anything wrong.

The IP address or range 3.238.0.0/16 has been blocked (disabled) by ‪ST47‬ for the following reason(s):

The IP address that you are currently using has been blocked because it is believed to be a web host provider. To prevent abuse, web hosts may be blocked from editing Wikipedia.
You will not be able to edit Wikipedia using a web host provider because it hides your IP address, much like a proxy or VPN.
We recommend that you attempt to use another connection to edit. For example, if you use a proxy or VPN to connect to the internet, turn it off when editing Wikipedia. If you edit using a mobile connection, try using a Wi-Fi connection, and vice versa. If you are using a corporate internet connection, switch to a different Wi-Fi network. If you have a Wikipedia account, please log in.
If you do not have any other way to edit Wikipedia, you will need to request an IP block exemption.

How to appeal if you are confident that your connection does not use a web host's IP address:
If you are confident that you are not using a web host, you may appeal this block by adding the following text on your talk page: ((unblock|reason=Caught by a web host block but this host or IP is not a web host. My IP address is _______. Place any further information here. ~~~~)). You must fill in the blank with your IP address for this block to be investigated. Your IP address can be determined here. Alternatively, if you wish to keep your IP address private you can use the unblock ticket request system. There are several reasons you might be editing using the IP address of a web host (such as if you are using VPN software or a business network); please use this method of appeal only if you think your IP address is in fact not a web host.

Administrators: The IP block exemption user right should only be applied to allow users to edit using web host in exceptional circumstances, and they should usually be directed to the functionaries team via email. If you intend to give the IPBE user right, a CheckUser needs to take a look at the account. This can be requested most easily at SPI Quick Checkuser Requests. Unblocking an IP or IP range with this template is highly discouraged without at least contacting the blocking administrator.

This block will expire on 05:34, 21 January 2026.

Even when blocked, you will usually still be able to edit your user talk page, as well as email administrators and other editors.

For information on how to proceed, please read the FAQ for blocked users and the guideline on block appeals. The guide to appealing blocks may also be helpful.

Other useful links: Blocking policy · Help:I have been blocked
Your IP address is in a range that has been blocked on all Wikimedia Foundation wikis. The block was made by ‪Jon Kolbert‬. The reason given is Open proxy/Webhost: See the help page if you are affected .
- Start of block: 05:35, 21 February 2024
- Expiry of block: 05:35, 21 February 2029
Your current IP address is 3.238.64.200. The blocked range is 3.238.0.0/16. Please include all above details in any queries you make. If you believe you were blocked by mistake, you can find additional information and instructions in the No open proxies global policy. Otherwise, to discuss the block please post a request for review on Meta-Wiki. You could also send an email to the stewards VRT queue at stewards@wikimedia.org including all above details.

You can view and copy the source of this page:

((Short description|American AI safety researcher))
((Use mdy dates|date=November 2023))((Infobox scientist
| workplaces        = ((ubl|[[OpenAI]]|
[[Alignment Research Center]]))
| alma_mater        = ((ubl|[[Massachusetts Institute of Technology]] (BS)|
[[University of California, Berkeley]] (PhD)))
| known_for         = ((ubl|[[AI alignment]]|
[[Reinforcement learning from human feedback]]))
| website           = ((url|https://paulfchristiano.com/))
))
'''Paul Christiano''' is an American researcher in the field of [[artificial intelligence]] (AI), with a specific focus on [[AI alignment]], which is the subfield of [[AI safety]] research that aims to steer AI systems toward human interests.<ref name=":0" /> He formerly led the language model alignment team at [[OpenAI]] and became founder and head of the non-profit [[Alignment Research Center]] (ARC), which works on theoretical AI alignment and evaluations of [[machine learning]] models.<ref>((Cite web |last=Piper |first=Kelsey |date=2023-03-29 |title=How to test what an AI model can — and shouldn't — do |url=https://www.vox.com/future-perfect/2023/3/29/23661633/gpt-4-openai-alignment-research-center-open-philanthropy-ai-safety |access-date=2023-08-04 |website=Vox |language=en))</ref><ref name=":1">((Cite news |last=Henshall |first=Will |date=September 7, 2023 |title=Paul Christiano – Founder, Alignment Research Center |work=[[TIME (magazine)|TIME magazine]] |url=https://time.com/collection/time100-ai/6309030/paul-christiano/ |access-date=2023-11-16))</ref> In 2023, Christiano was named as one of the [[Time 100|''TIME'' 100]] Most Influential People in AI (''TIME''100 AI).<ref name=":1" /><ref>((Cite news |last=Sibley |first=Jess |date=September 10, 2023 |title=The Future Is Now |volume=202 |work=[[Time (magazine)|Time magazine]] |issue=11/12 |url=https://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=172374416&lang=en-gb&site=eds-live&scope=site |access-date=2023-11-16 |via=[[EBSCOHost]]))</ref>

In September 2023, Christiano was appointed to the UK government's Frontier AI Taskforce advisory board.<ref>((Cite news |last=Skelton |first=Sebastian Klovig |date=7 September 2023 |title=Government AI taskforce appoints new advisory board members |work=ComputerWeekly.com |url=https://www.computerweekly.com/news/366551256/Government-AI-taskforce-appoints-new-advisory-board-members |access-date=2023-11-16))</ref> He is also an initial trustee on [[Anthropic]]'s Long-Term Benefit Trust.<ref name=":3">((Cite news |last=Matthews |first=Dylan |date=25 September 2023 |title=The $1 billion gamble to ensure AI doesn’t destroy humanity |work=[[Vox (website)|Vox]] |url=https://www.vox.com/future-perfect/23794855/anthropic-ai-openai-claude-2 |access-date=2023-11-16))</ref>

==Education== 
Christiano attended the [[Harker School]] in San Jose, California.<ref name=":4">((Cite web |last=Kehoe |first=Elaine |date=October 2008 |title=Mathematics People – 2008 International Mathematical Olympiad |url=https://www.ams.org/notices/200810/tx081001284p.pdf |access-date=2023-11-16 |website=American Mathematical Society))</ref> He competed on the U.S. team and won a silver medal at the 49th [[International Mathematical Olympiad|International Mathematics Olympiad]] (IMO) in 2008.<ref name=":4" /><ref>((Cite journal |last=Feng |first=Zumin |last2=Gelca |first2=Razvan |last3=Le |first3=Ian |last4=Dunbar |first4=Steven R. |date=June 2009 |title=NEWS AND LETTERS: 49th International Mathematical Olympiad |url=https://www.jstor.org/stable/27765911 |journal=Mathematics Magazine |volume=82 |issue=e |pages=235–238 |jstor=27765911))</ref> 

In 2012, Christiano graduated from the [[Massachusetts Institute of Technology]] (MIT) with a degree in mathematics.<ref>((Cite web |title=Paul F. Christiano |url=https://dl.acm.org/profile/81485658302 |access-date=2023-11-16 |website=Association for Computing Machinery Digital Library))</ref><ref name=":2" /> At MIT, he researched data structures, quantum cryptography, and combinatorial optimization.<ref name=":2">((cite web |title=About the Authors: Theory of Computing: An Open Access Electronic Journal in Theoretical Computer Science |url=https://theoryofcomputing.org/articles/v009a009/about.html |access-date=2023-11-16))</ref> 

He then went on to complete a [[Doctor of Philosophy|PhD]] at the [[University of California, Berkeley]].<ref>((Cite web |last= |first= |title=Paul Christiano – Research Associate |url=http://www.fhi.ox.ac.uk/ |access-date=2023-08-04 |website=The Future of Humanity Institute |language=en-GB))</ref> While at Berkeley, Christiano collaborated with researcher [[Katja Grace]] on AI Impacts, co-developing a preliminary methodology for comparing supercomputers to brains, using traversed edges per second (TEPS).<ref>((Cite news |last=Hsu |first=Jeremy |date=26 August 2015 |title=Estimate: Human Brain 30 Times Faster than Best Supercomputers |work=[[IEEE Spectrum]] |url=https://spectrum.ieee.org/estimate-human-brain-30-times-faster-than-best-supercomputers |access-date=2023-11-16))</ref> He also experimented with putting [[Carl Shulman|Carl Shulman]]'s donor lottery theory into practice, raising nearly $50,000 in a pool to be donated to a single charity.<ref>((Cite news |last=Paynter |first=Ben |date=January 31, 2017 |title=Take A Chance With Your Charity And Try A Donor Lottery |work=[[Fast Company]] |url=https://www.fastcompany.com/3067596/take-a-chance-with-your-charity-and-try-a-donor-lottery |access-date=2023-11-16))</ref>

==Career==
At OpenAI, Christiano co-authored the paper "Deep Reinforcement Learning from Human Preferences" (2017) and other works developing [[reinforcement learning from human feedback]] (RLHF).<ref>((Cite journal |last1=Christiano |first1=Paul F |last2=Leike |first2=Jan |last3=Brown |first3=Tom |last4=Martic |first4=Miljan |last5=Legg |first5=Shane |last6=Amodei |first6=Dario |date=2017 |title=Deep Reinforcement Learning from Human Preferences |url=https://proceedings.neurips.cc/paper_files/paper/2017/hash/d5e2c0adad503c91f91df240d0cd4e49-Abstract.html |journal=Advances in Neural Information Processing Systems |publisher=Curran Associates, Inc. |volume=30))</ref><ref>((Cite journal |last1=Ouyang |first1=Long |last2=Wu |first2=Jeffrey |last3=Jiang |first3=Xu |last4=Almeida |first4=Diogo |last5=Wainwright |first5=Carroll |last6=Mishkin |first6=Pamela |last7=Zhang |first7=Chong |last8=Agarwal |first8=Sandhini |last9=Slama |first9=Katarina |last10=Ray |first10=Alex |last11=Schulman |first11=John |last12=Hilton |first12=Jacob |last13=Kelton |first13=Fraser |last14=Miller |first14=Luke |last15=Simens |first15=Maddie |date=2022-12-06 |title=Training language models to follow instructions with human feedback |url=https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html |journal=Advances in Neural Information Processing Systems |language=en |volume=35 |pages=27730–27744|arxiv=2203.02155 ))</ref> He is considered one of the principal architects of RLHF,<ref name=":1" /><ref name=":3" /> which in 2017 was "considered a notable step forward in AI safety research", according to ''[[The New York Times]]''.<ref>((Cite news |last=Metz |first=Cade |date=August 13, 2017 |title=Teaching A.I. Systems to Behave Themselves |work=[[The New York Times]] |url=https://www.nytimes.com/2017/08/13/technology/artificial-intelligence-safety-training.html |url-access=subscription |access-date=2023-11-16))</ref> Other works such as "AI safety via debate" (2018) focus on the problem of ''scalable oversight'' – supervising AIs in domains where humans would have difficulty judging output quality.<ref>((Cite arXiv |last1=Irving |first1=G. |last2=Christiano |first2=P. |last3=Amodei |first3=Dario |date=2018-05-02 |title=AI safety via debate |class=stat.ML |eprint=1805.00899 ))</ref><ref>((Cite arXiv |last1=Wu |first1=Jeff |last2=Ouyang |first2=Long |last3=Ziegler |first3=Daniel M. |last4=Stiennon |first4=Nissan |last5=Lowe |first5=Ryan |last6=Leike |first6=J. |last7=Christiano |first7=P. |date=2021-09-22 |title=Recursively Summarizing Books with Human Feedback |class=cs.CL |eprint=2109.10862))</ref><ref>((Cite arXiv |last1=Christiano |first1=P. |last2=Shlegeris |first2=Buck |last3=Amodei |first3=Dario |date=2018-10-19 |title=Supervising strong learners by amplifying weak experts |class=cs.LG |eprint=1810.08575))</ref>

Christiano left OpenAI in 2021 to work on more conceptual and theoretical issues in AI alignment and subsequently founded the [[Alignment Research Center]] to focus on this area.<ref name=":0">((Cite web |title=A.I. has a '10 or 20% chance' of conquering humanity, former OpenAI safety researcher warns |url=https://fortune.com/2023/05/03/openai-ex-safety-researcher-warns-ai-destroy-humanity/ |access-date=2023-06-04 |website=Fortune |language=en))</ref> One subject of study is the problem of ''eliciting latent knowledge'' from advanced machine learning models''.''<ref>((Cite arXiv|last1=Burns |first1=Collin |last2=Ye |first2=Haotian |last3=Klein |first3=Dan |last4=Steinhardt |first4=Jacob |date=2022 |title=Discovering Latent Knowledge in Language Models Without Supervision |class=cs.CL |eprint=2212.03827))</ref><ref>((Cite web |last1=Christiano |first1=Paul |last2=Cotra |first2=Ajeya |last3=Xu |first3=Mark |date=December 2021 |title=Eliciting Latent Knowledge: How to tell if your eyes deceive you |url=https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8/edit?usp=embed_facebook |access-date=2023-04-16 |website=Google Docs |publisher=Alignment Research Center |language=en))</ref> ARC also develops techniques to identify and test whether an AI model is potentially dangerous.<ref name=":1" /> In April 2023, Christiano told ''[[The Economist]]'' that ARC was considering developing an industry standard for AI safety.<ref>((Cite news |date=2023-04-19 |title=How generative models could go wrong |work=[[The Economist]] |url=https://www.economist.com/science-and-technology/2023/04/19/how-generative-models-could-go-wrong |access-date=2023-11-16))</ref>

=== Views on AI risks ===
He is known for his views on the potential risks of advanced AI. In 2017, [[Wired (magazine)|''Wired'' magazine]] stated that Christiano and his colleagues at OpenAI weren't worried about the destruction of the human race by "evil robots", explaining that "[t]hey’re more concerned that, as AI progresses beyond human comprehension, the technology’s behavior may diverge from our intended goals."<ref>((Cite news |last=Newman |first=Lily Hay |date=September 2017 |title=Should We Worry? – Will AI Turn Against Me? |work=[[Wired (magazine)|Wired]] |url=https://www.wired.com/2017/08/dont-worry-be-happy/ |access-date=2023-11-16))</ref>

However, in a widely quoted interview with ''[[Business Insider]]'' in 2023, Christiano said that there is a “10–20% chance of AI takeover, [with] many [or] most humans dead.” He also conjectured a “50/50 chance of doom shortly after you have AI systems that are human level.”<ref>((Cite web |last=Nolan |first=Beatrice |title=Ex-OpenAI researcher says there's a 50% chance AI development could end in 'doom' |url=https://www.businessinsider.com/openai-researcher-ai-doom-50-chatgpt-2023-5 |access-date=2023-06-04 |website=Business Insider |language=en-US))</ref><ref name=":0" />

== Personal life ==
Christiano is married to Ajeya Cotra of [[Open Philanthropy]].<ref>((Cite news |last=Piper |first=Kelsey |date=June 2023 |title=A Field Guide to AI Safety |work=Asterisk Magazine |issue=3 |url=https://asteriskmag.com/issues/03/a-field-guide-to-ai-safety |access-date=2023-11-16))</ref>

== References ==
((reflist))

==External links==
* [https://paulfchristiano.com/ Personal website]
((Authority control))
[[Category:Theoretical computer scientists]]
[[Category:Year of birth missing (living people)]]
[[Category:Living people]]
[[Category:Place of birth missing (living people)]]
[[Category:Nationality missing]]
[[Category:Massachusetts Institute of Technology alumni]]
[[Category:University of California, Berkeley alumni]]
[[Category:OpenAI]]

Pages transcluded onto the current version of this page (help):

Paul Christiano (researcher) (edit)
Template:Authority control (view source) (template editor protected)
Template:Br separated entries (view source) (template editor protected)
Template:Cite arXiv (view source) (template editor protected)
Template:Cite journal (view source) (protected)
Template:Cite news (view source) (protected)
Template:Cite web (view source) (protected)
Template:Count (view source) (template editor protected)
Template:DMCA (view source) (template editor protected)
Template:Dated maintenance category (view source) (template editor protected)
Template:Dated maintenance category (articles) (view source) (template editor protected)
Template:FULLROOTPAGENAME (view source) (template editor protected)
Template:Hlist/styles.css (view source) (protected)
Template:If empty (view source) (protected)
Template:Infobox (view source) (template editor protected)
Template:Infobox person (view source) (template editor protected)
Template:Infobox scientist (view source) (template editor protected)
Template:Main other (view source) (protected)
Template:Ns has subpages (view source) (protected)
Template:PAGENAMEBASE (view source) (template editor protected)
Template:Pagetype (view source) (protected)
Template:Plainlist/styles.css (view source) (protected)
Template:Pluralize from text (view source) (template editor protected)
Template:Reflist (view source) (protected)
Template:Reflist/styles.css (view source) (protected)
Template:SDcat (view source) (protected)
Template:Short description (view source) (protected)
Template:Short description/lowercasecheck (view source) (protected)
Template:Template other (view source) (protected)
Template:URL (view source) (template editor protected)
Template:Ubl (view source) (template editor protected)
Template:Unbulleted list (view source) (template editor protected)
Template:Url (view source) (template editor protected)
Template:Use mdy dates (view source) (template editor protected)
Template:Wikidata image (view source) (template editor protected)
Module:Arguments (view source) (protected)
Module:Authority control (view source) (template editor protected)
Module:Authority control/config (view source) (template editor protected)
Module:Check for clobbered parameters (view source) (template editor protected)
Module:Check for unknown parameters (view source) (protected)
Module:Citation/CS1 (view source) (protected)
Module:Citation/CS1/COinS (view source) (protected)
Module:Citation/CS1/Configuration (view source) (protected)
Module:Citation/CS1/Date validation (view source) (protected)
Module:Citation/CS1/Identifiers (view source) (protected)
Module:Citation/CS1/Utilities (view source) (protected)
Module:Citation/CS1/Whitelist (view source) (protected)
Module:Citation/CS1/styles.css (view source) (protected)
Module:Detect singular (view source) (template editor protected)
Module:Disambiguation/templates (view source) (protected)
Module:EditAtWikidata (view source) (protected)
Module:If empty (view source) (protected)
Module:Infobox (view source) (template editor protected)
Module:Infobox/styles.css (view source) (template editor protected)
Module:InfoboxImage (view source) (template editor protected)
Module:List (view source) (protected)
Module:Navbar (view source) (protected)
Module:Navbar/configuration (view source) (protected)
Module:Navbox (view source) (template editor protected)
Module:Navbox/configuration (view source) (template editor protected)
Module:Navbox/styles.css (view source) (template editor protected)
Module:Ns has subpages (view source) (protected)
Module:Pagetype (view source) (protected)
Module:Pagetype/config (view source) (protected)
Module:Pagetype/disambiguation (view source) (protected)
Module:Pagetype/setindex (view source) (protected)
Module:Pagetype/softredirect (view source) (protected)
Module:ParameterCount (view source) (template editor protected)
Module:SDcat (view source) (protected)
Module:Separated entries (view source) (protected)
Module:String (view source) (protected)
Module:TableTools (view source) (protected)
Module:Template wrapper (view source) (template editor protected)
Module:Text (view source) (template editor protected)
Module:URL (view source) (template editor protected)
Module:Unsubst (view source) (protected)
Module:Wikitext Parsing (view source) (protected)
Module:Yesno (view source) (protected)

Return to Paul Christiano (researcher).

Retrieved from "https://en.wikipedia.org/wiki/Paul_Christiano_(researcher)"