copyright notice
link to the published version: IEEE Computer, September, 2023, archive copy

accesses since August 22, 2023

Fatal Flaws in ChatAI as a Content Generator

Hal Berghel

ABSTRACT: Unfortunately, the current fascination with AIChat focuses on content generation which may be its weakest suit.

There is a lot to like about AIChat. For one thing, it satisfies human curiosities - especially relating to some of Alan Turing's observations about the potential for machine intelligence. In general, and despite religious proclamations to the contrary, curiosity is a great motivator. [1] As Nietzsche put it “what does not kill me makes me stronger.” [2]] What is more, AIChat is fun! But beyond that, it may it may ultimately prove valuable in re-purposing codebases, as a component of database query support languages, extending the capabilities of scene analyzers and pattern recognizers, getting the sense of the contents of massive databases, etc. Large language data models have already shown themselves as useful utilities for suggesting linguistic alternatives, additional contexts, viewpoints, facts, etc. in text processing. It would also seem to have great potential in natural language translation. It is unfortunate, however, that the current fascination with AIChat is automated content generation, for this is one application in which AIChat has the least to offer. As we suggested elsewhere, [9] although AIChat generally, and ChatAI in particular, may be said to pass a limited Turing Test given some important caveats, this may say more about the limits of Turing Tests than it does about the strength of ChatAI. I look at AIChat output as a major step below hastily authored polemics, self-published books, and gratuitous, vanity Web sites. In fact, to carry this analogy further, there is a sense in which AIChat is vanity publishing at its worst: where any vainglory lingers in the anonymity of the cloud. In terms of content, AIChat is better thought of as bloviation generation. As such it is likely to contribute little to edification and scholarship, though it will have profound effects on social media and partisan politics.

So what is wrong with AIChat? Why will it be of limited value to scholarship? In short, AIChat has some serious design flaws. For one, AIChat fails to advance interactive technology much beyond the more pedestrian goals of content generation. It's masterfully implemented, to be sure, but impoverished when viewed in the light of the visions of futurists like Vanavar Bush, Doug Engelbart, and Ted Nelson. Such visions (e.g., Bush's memex, Engelbart's oN-Line System, and Nelson's Xanadu) were focused on achieving worthy, substantial intellectual goals, and not shallow, uninformed, pedestrian activities. I expect that Ted Nelson (the last of these visionaries standing at this writing) is nonplussed by AIChat's feeble attempt to advance our collective intellect. In fact, the deficiencies of AIChat in fact parallel those of the World Wide Web. This is no coincidence.


The World Wide Web was spawned by the work of Tim Berners Lee at the international particle physics laboratory, CERN, in Geneva, Switzerland, and significantly advanced by the Mosaic browser software developed by Marc Andreessen and Eric Bina at the National Center for Supercomputing Applications at the University of Illinois in 1992. [10] The defining characteristic of the Web was twofold:

  1. the deployment of the new platform-independent, application layer, communication protocol called HTTP that supported extensible, distributed, collaborative hypermedia, and
  2. the integration of a variation of the SGML tagged image format language standard called HTML, that integrated hypertext into documents/Web pages.

These two contributions laid the groundwork for the open, interactive Internet information system that we call the World Wide Web. As such, the Web may be thought of as a paradigm shift away from earlier, more insular, application-layer protocols.

The idea of using a tagged image format for markup languages preceded the development of HTML by several decades. Both SGML and HTML descended from IBM's GML language standard defined in the late 1960s. Certainly, the addition of hyperlinks into HTML was noteworthy. Hyperlinks enabled network resources to be accessed easily through Uniform Resource Locators (URLs). Thus, the use of hyperlinks in HTML took the GML concept one step closer to the vision of linking together the world's information resources. Browsers added considerably to the value of the Web by supporting robust media rendering and providing convenient techniques to access Web resources via a mouse click on sensitized and clearly identified links in rendered media.

For these reasons, the Web became a primary enabling technology on the Internet, quickly taking its rightful place along with earler “killer apps” like Telnet, FTP, and email.


So how far did the Web go in realizing the visions of Bush, Engelbart, and Nelson? In short – and this is the most important point – not nearly as far as the hype and hyperbole would suggest. The advantages of the Web over earlier applications-layer protocols included: (a) open architecture, (b) platform and media-independency, (c) native web-compliant browser support for convenient Internet navigation and robust media rendering, and (d) the use of unidirectional hyperlinks. These advantages alone are not the stuff of which cyber-visions are made.

While it certainly must be admitted that the web constituted a considerable practical advance in Internet information access and delivery systems, it was conceptually primitive when compared to the visions of Bush, Engelbart, and Nelson. For one, the implementation of the hyperlinks was crude. As implemented on the Web, hyperlinks involved drawing file-oriented chunks of media to the browser for independent rendering as distinct entities. There was (and remains for the most part) no correlative capacity for integration and annotation. So where you might have started with one Web page, after clicking on a sensitizes hyperlink, and through the magic of HTTP and HTML, now you have two web pages. This linear accumulation of independent resources is at most a minor technological leap toward information absorption. While the Web was comparatively a major advance in network media sharing over FTP and Telnet, it falls fall short of the visions that Bush, et al, had in mind.

Our ineptitude in getting at the record is largely caused by the artificiality of systems of indexing…. The human mind does not work that way. It operates by association . With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain.... Selection by association, rather than indexing, may yet be mechanized.

A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory…. [memex provides] associative indexing, the basic idea of which is a provision whereby any item may be caused at will to select immediately and automatically another….

Before him are the two items to be joined, projected onto adjacent viewing positions. At the bottom of each there are a number of blank code spaces, and a pointer is set to indicate one of these on each item. The user taps a single key, and the items are permanently joined

Thereafter, at any time, when one of these items is in view, the other can be instantly recalled … Moreover, when numerous items have been thus joined together to form a trail, they can be reviewed in turn…. It is exactly as though the physical items had been gathered together from widely separated sources and bound together to form a new book. It is more than this, for any item can be joined into numerous trails…Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items. Occasionally he inserts a comment of his own, either linking it into the main trail or joining it by a side trail to a particular item. [3]

It is important to remember that Bush wrote this in 1945! – before commercial computers, before the Internet, before text and word processing programs, and before the formal study of information retrieval. But it is fairly easy to see that Bush is committed to the principle of bringing together concepts and ideas which goes far beyond linking static documents. The trails of which he speaks are integrated, annotated threads of fragments from many sources, carefully woven into a new creative artifact. Note that this does not involve the linear transversal of a sequence of hyperlinked documents currently provided by the Web – which falls quite short of his vision. One does not create an original manuscript by assembling a collection of old readings.


Flash forward twenty years and Ted Nelson introduces the world to hypertext – not the hobbled form that we know from the World Wide Web – but a more full-bodied form implied by Bush. Nelson's planned version of hypertext directly supports content transclusion and copyright management still absent from the Web.

If a writer is really to be helped by an automated system, it ought to do more than retype and transpose: it should stand by him during the early periods of muddled confusion, when his ideas are scraps, fragments, phrases, and contradictory overall designs. And it must help him through to the final draft with every feasible mechanical aid-- making the fragments easy to find, and making easier the tentative sequencing and juxtaposing and comparing. [5]

He expands his idea with an overview of the overall design that a worthy information sharing system which Nelson named ELF (evolutionary file system) should have:

  1. support of hyperlinks to network file resources
  2. informal, not rigid, formal file relations – it must store media in any form and arrangement desired
  3. all files must be able to hold associated commentaries and explanations
  4. the files and file elements must be modifiable at will, and such modifications must be “dynamic” in that the change of one element will automatically change other associated elements
  5. the files must be retained in version controlled environment
  6. all element links must be bi-directional
  7. organization of data elements must support nonlinearity

We note that only 1. is supported by the Web – and even then in only limited form, for (a) Web hyperlinks are unidirectional, and (b) most links remain URLs, and not URNs (resource names) or URIs (resource identifiers). One sees a clearer image of Nelson's ambitions in his definition of hypertext as a “ body of written or pictorial material interconnected in such a complex way that it could not conveniently be presented or represented on paper. It may contain summaries, or maps of its contents and their interrelations; it may contain annotations, additions and footnotes from scholars who have examined it [and leading to] is a bundle of relationships subject to all kinds of twists, inversions, involutions and arrangement: these changes are frequent but unpredictable.” [misspellings corrected] Obviously, Nelson has in mind a data structure that supports collaboration, specifically including trans-document interconnectivity which he elsewhere calls transclusion. [6] His intentions, which are consistent with Bush's original vision, are clearly seen in Figures 2-4 in his articles.

Nelson, like Bush, is short on technical details in his publications. However, if computer scientists take the time to seriously study these contributions, they will see that the scope of their collective visions (along with Engelbart's [7]) circumscribe a much more expansive view of information retrieval (IR) that spans networking, data processing, text and word processing, and multimedia. The over-arching idea is that if human-centered IR must be driven by the intellectual requirements of the users. In modern terms we might label this the search for a grand unification of all cybermedia under a robust set of content-oriented, resource-level protocols. In any case, such requirements must efficiently and effectively advance human creativity in all cognitive areas: including science, technology, art, humanities, etc. While the articles may seem obtuse by modern standards, the terminology archaic (at least in terms of computing history), and the descriptions and definitions awkward by modern standards, the underlying concepts remain innovative, radical and consequential to the present day – well over a half-century after they were advanced.


Doug Engelbart summarizes the goal of the collective Bush-Engelbart-Nelson (BEN) vision nicely in his companion to Nelson's 1995 update. [4] What is needed, he offers, is a tool that focuses on a universal knowledge base rather than function-oriented paradigms like word processors, file transfer protocols – and even the present manifestation of hypertext! The differences with existing systems becomes clear in his discussion of relevant data structures:

Every knowledge object—from the largest documents, to aggregate branches, down to content units such as characters—has an unambiguous address, understandable and readable by a user, and referenceable anywhere in the hyperdocument system…..A structured, mixed-object hyperdocument may be displayed with a flexible choice of viewing options: selective level clipping, filtering on content, truncation or other transformation of object content, new sequences or groupings of objects including those residing in other documents, etc. Links may specify views so traversal retrieves the destination object with a prespecified presentation view (e.g., display as a high-level outline or display only a particular statement). View specification becomes a natural and constantly employed part of a user's vocabulary.

It is clear that both Engelbart and Nelson advance the idea that the current Web, browsers, and media-manipulating technologies should be seen as merely a first step toward a much more ambitious cyber-media-oriented end state. Two mission-critical features necessary in realizing this potential are Nelson's transclusion and transcopyright. Transclusion is particularly interesting as it involves “reuse with original context available, through embedded shared instancing” that span any arbitrary number of sources and at an arbitrary level of element size within artifacts. The operational principle is the integration of contexts in the presentation and manipulation of media. The concept is easier to understand with one of Nelson's demonstrations. [7]

The many talks and demonstrations provided by Nelson and Engelbart collectively circumscribe the virtual corpus of their vision – or, more accurately, the vision they shared with Vannevar Bush. The term ‘virtual' must be emphasized because neither was able to bring their vision to commercial fruition, which is both a measure of the complexity of the task as well as limited immediate business appeal. Such was our experience with the enterprises of detecting gravitational waves and proving Fermat's Last Theorem. One must remember that the challenge of implementing a namespace for consistent and complete Uniform Resource Names (URNs) proved so daunting that the entire enterprise was abandoned in 2005. The fabrics of transclusion and transcopyright infrastructures are of a distinctly greater complexity than even URNs. The evolution of the Web after the usurpation of control by corporate interests is a history of picking the lowest-hanging and easiest to implement Internet fruit with the greatest commercial potential (compare the fates of cookies and shopping carts vs. the do-not-track HTTP header). This is not to deny the enormous contributions of the Internet and the World Wide Web in support of knowledge workers. But both, and the Web in particular, are only instruments to advance connectivity to global digital resources. They are passive with respect to advancing human intellect. They connect people with resources, plain and simple. Some Internet resources are noble and worthy while others are unfit for human consumption. The Web is agnostic in this regard.


Bush, Nelson and Engelbart were all about the use of technology to increase the efficiency and effectiveness of human intelligence – as opposed to the development of non-human intelligence. Of course using digital technology to interconnect digital resources, and build thought swarms and collaboratories on the back of non-linear media traversal and independently of the logical structure of the forms of media, and so forth, were one part of this mix. But only a part. But there is much, much more involved with the process of augmenting human intelligence than networking computer systems, just as there is much more to text processing than the cut-copy-paste desktop metaphor and the rapid cursor movement technologies. Doug Engelbart, in particular, understood this point as he made contributions to both the visionary and mundane. [11] [12] The working model of the vision must involve the creation of an environment that facilitates the creative enterprise, but the model environment is not the end goal. That's the major deficiency with the content generation aspect of AIChat which involves a retro dorsal attachment to the static, Web/Internet model of resource interconnectivity rather than the vision of intellectual augmentation proposed by Bush, Nelson, and Engelbart.

To illustrate, if a serious commitment was made by AIChat to enhance human intellect, the absence of source identification and credit (as in Nelson's transcopyright) should have been an immediate deal breaker. The idea that human knowledge may be advanced by purloining anonymous content from undisclosed data repositories, with or without the use of large language neural networks, is preposterous. This is not ‘standing on the shoulders of giants,' but more akin to wallowing in the muck and mire with lower life forms. This is precisely what is wrong with non-vetted recommender systems: without knowing the source of recommendations, there is no way to assign credibility and value to the recommendations. This is also the primary reason to challenge any lingering faith in the wisdom of crowds. [13]

And so it is with non-vetted automated content generation. What are the sources of this content? Absent, identification, documentation, and context, such content is simply automated. anonymized blather. What could be wrong with building content generation on that?

The underlying philosophy of AIChat is inherently anti-scholarly and predicated on the principle that the value of content to an audience is determined by tribal serviceability. We observe that this is one legacy of partisan politics as we know it. It should surprise no one that AIChat will prove to be extremely popular in that realm.

Will AIChat (qua automated bloviation) become a displacing technology? If history is any measure, the opportunity to advance content-light, undocumented, and uncredited opinion to a premier position in mass communication is a temptation too great to be passed over by manipulators of the public, so the likelihood is that the answer will be affirmative. By its very nature, resulting communication will be convergent rather than divergent, and it is unlikely to produce any great innovations or unique capabilities.

On the other hand, although AIChat is arguably sub-cerebral knowledge work, it has obvious, residual potential regarding information management – specifically in terms of applications, interface support, information retrieval, code-repurposing and the like.


[1] J. Agar, 2016 Wilkins–Bernal–Medawar lecture The curious history of curiosity-driven research, Notes and Records: The Royal Society Journal of the History of Science, v. 71, pp. 409-429, 2017. (available online: )

[2] K. Stoner, Science proves that what doesn't kill you makes you stronger, Northwestern Now (online), October 1, 2019. (available online: )

[3] V. Bush, “As we may think,” Atlantic Monthly, vol. 176, pp. 101–108, Jul. 1945. {available online: )

[4] D. Engelbart and W. English, A research center for augmenting human intellect , in 1968 Fall Joint Computer Conference, AFIPS, San Francisco, pp. 395-410, 1968. (available online: )

[5] T. Nelson, “Complex information processing: A file structure for the complex, the changing and the indeterminate,” in Proc. ACM 20 th Nat. Conf., Aug. 1965, pp. 84–100. (available online: )

[6] T. Nelson, The heart of connection: hypermedia unified by transclusion, Communications of the ACM, 38:8, pp. 31-33, 1995. (available online: )

[7] T. Nelson, Ted Nelson Demonstrates XanaduSpace, YouTube, Sep 6, 2008. (available online: )

[8] D. Engelbart, Toward augmenting the human intellect and boosting our collective IQ, Communications of the ACM, 38:8, pp. 30-32, 1995. (available online: )

[9] H. Berghel, ChatGPT and AIChat Epistemology, Computer , 56:5, pp. 130-137, 2023. (available online: )

[10] H. Berghel and Douglas Blank, The World Wide Web, in M. Zelkowitz (ed) Advances in Computers, Elesvier, Volume 48, pp. 179-218, 1999, doi: 10.1016/S0065-2458(08)60020-0. (available online: )

[11] D. Engelbart, “The Mother of All Demos,” demonstration hosted by the ACM/IEEE-CS Fall Joint Computer Conference, San Francisco, December 9, 1968. (archive available online: )

[12] A. Fisher, How Doug Engelbart Pulled off the Mother of All Demos, Wired, Dec 9, 2018. (available online: )

[13] H. Berghel, Social Media and the Banality of (Online) Crowds, Computer, 55:11, 2022. (available online: )