Testing ChatGPT against an editor’s rewrite

Some of the results I’ve seen from OpenAI’s ChatGPT (https://openai.com/blog/chatgpt/) have been astounding. Scary, but astounding nonetheless.

To test it out from a rewriting/rewording/editing perspective, I decided to feed it a piece of wordy text I edited some years ago. I copied in the original text and within about 20 seconds (maybe less) ChatGPT gave me its version, which is certainly easier to read and gives more information than the original even though it’s longer. For comparison, I’ve also included my edited version of that original text below. My task was to edit what was there, not add more information to the author’s original text, which is likely why mine is much shorter. However, mine took a few minutes to edit, whereas ChatGPT’s took less than 20 seconds.

For my first test, I just copied in the original text and pressed the arrow button for ‘go’. ChatGPT has no options except in how you frame your request. When I changed the request by adding Simplify: in front of the original text, I got a different and much shorter result. And different results again when I added Explain this to a five year old: or Write this as a senior student: in front of the text.

As with any AI technology like this, you would need to check the resulting text to see if it still conveyed your message correctly. And that the spelling suited your form of English (I did not test any other languages), and that the content it provided was accurate.

Original text from the author (105 words, 644 characters)

Another climate change related issue pertaining to individual MPAs where marine planning is pertinent, partly as it concerns areas outside of MPAs, is a required change in boundary or even in site location in response to climate change. Whilst it may seem premature to raise the potential for such changes when the process of identifying let alone designating all sites and providing conservation advice is still underway, it is appropriate when looking ahead over a 20 year (or more) time horizon. Indeed it is understandable why there is such a ‘gap’ in the current process given the amount of requirements and considerations to work through.

After my editing (19 words, 118 characters)

In relation to MPAs and marine planning, climate change effects may result in future boundary or site location changes

ChatGPT’s rewording of the original text (173 words, 1126 characters)

It is important to consider the potential impacts of climate change on marine protected areas (MPAs) and to plan for potential changes in boundary or site location in response. Climate change is already having significant impacts on the world’s oceans, including rising sea levels, ocean acidification, and changes in ocean circulation patterns. These changes can affect the habitats and species found within MPAs, as well as the effectiveness of the MPA in conserving these species and habitats.

To address this issue, MPA managers and planners can incorporate climate change considerations into the design and management of MPAs. This can include identifying and prioritizing climate change-resilient sites for MPA designation, and developing adaptive management strategies that can be implemented if and when the boundaries of an MPA need to be adjusted in response to climate change impacts. Additionally, it is important to regularly monitor and assess the impacts of climate change on MPAs and the species and habitats they protect, in order to inform management decisions and ensure the ongoing effectiveness of the MPA.

ChatGPT’s rewording after adding ‘Simplify:’ in front of the request (53 words, 325 characters)

Climate change may require changes to the boundaries or locations of marine protected areas (MPAs), and it is important to consider these potential changes when planning for the future. The current process for identifying and designating MPAs may not address this issue, but it is important to consider it over the long term.

ChatGPT’s rewording after adding ‘Explain this to a five year old:’ in front of the request (117 words, 602 characters)

Climate change may cause the boundaries of places in the ocean where marine life is protected to move or change. It is important to think about this when making plans for the future of these special places. The process of deciding where these places should be and how to protect them is still happening, so it may seem too early to talk about changes. But it is good to think about these things now so that we can make sure the marine life is protected for a long time. There are a lot of things to think about and do in this process, so it is not surprising that some things might be left out for now.

ChatGPT’s rewording after adding ‘Write this for senior student:’ in front of the request (121 words, 590 characters)

Climate change may necessitate changes to the boundaries or locations of marine protected areas (MPAs), and it is crucial to take these potential changes into account when planning for the future of these sites. While the process of identifying and designating MPAs is ongoing, it may be premature to discuss boundary shifts. However, given the long-term nature of climate change, it is important to consider the potential for such shifts over a time horizon of 20 years or more. The current process for identifying and designating MPAs may not adequately address this issue, resulting in a “gap” in conservation efforts. It is important to address this gap in order to ensure the effectiveness of MPAs in the face of climate change.

For more ways to structure queries, check some options here: https://medium.com/mlearning-ai/use-this-prompting-trick-to-improve-chatgpts-performance-aefd5d9ba0e3

For information about potential implications of this technology (no doubt there will be MANY articles like this in the next few weeks and months—ChatGPT was only opened to the public about 2 weeks ago (as at 12 December 2022):

• https://techcrunch.com/2022/12/09/is-chatgpt-a-virus-that-has-been-released-into-the-wild/
• https://www.theatlantic.com/technology/archive/2022/12/openai-chatgpt-writing-high-school-english-essay/672412/
• https://idratherbewriting.com/blog/chatgpt-automation-and-atrophy/ (long piece, but excellent points made about the potential effects of automation on writing using automated cars and driving as a simile)

Update 13 December 2022: This post (and others like it in recent days) has generated quite a lot of discussion in editors’ groups, on LinkedIn etc. (often about the future of working as an editor [aka ‘the sky is falling!’]). I’ve added below some of the responses I’ve made in those forums in relation to this post:

• On the issue of me feeding it text (as versus asking it to create text for me), I responded: Yes, I deliberately fed it the original. My ‘use case’ was on rewording/rewriting, not writing something from scratch. My test was to check how it would perform rewording wordy passages that took more time than usual for an editor to unpack. And this was one of them. In my situation, I work on corporate reports and some of the writers are less than clear. When your average hourly work rate gets hammered by a couple of paras that take 10+ mins to unpack, you often need to set them aside and come back to them later, or give up and write a comment asking the author to reword. Along with all sorts of other tools we use to speed up our work, I wanted to see if ChatGPT could offer a potential solution to these sorts of wordy paragraphs—reword in such a way that was clear, and perhaps more concise and possibly more precise. As with any tool, I believe that any such use still needs to be checked, but if it can stop my brain exploding with such passages, I can see it being something I may use on occasion.
• You are flagging a proof-reading role for the human technical writer/ editor following the AI’s processing step. Do you see any cases where the AI editor would not be able to perform as good as or better than a human? My response: It’s so new I haven’t really thought much yet about the implications of this AI tech on pretty much everything. My initial concerns would be around writing assignments that aren’t your own work and copyright and plagiarism issues at the school / university level; fake journalistic copy; potentially fake legal notices, medical info, safety warnings etc. (i.e. stuff that can have serious consequences). I have no idea if it can perform as well or better than a human editor. Because we don’t know its sources and can’t confirm the veracity of the information it writes, having human eyes go over the produced copy has to be part of the process. But I can see a time in the near future where it may be hard to verify what’s written once a lot of info is generated by AI. It’s hard enough to verify things now, when most has been written by humans. At a personal level, I’ve had some paragraphs that have taken far more time to unpack than they should, and this could be a ‘quick and dirty’ solution to that. And for reference, I only fed it that one passage, with no other context. It was from a UK report, though I’m not sure if that report was published internally or to the outside world.

[Link last checked December 2022]