Use computers as powerful as in 1985 or AI controls humans or ?

A way to prevent AGI from taking over or destroying humanity is to strictly limit the computing power used on unknown AI algorithms. My back of the envelope calculations[1] show that restricting the hardware to 64 KiB of total storage is definitely sufficient to prevent an independence gaining AGI, and restricting to 2 MiB of storage is very likely to be sufficient to prevent an indepence gaining AGI. State of the art AI on the other hand tends to be using at least 1 GiB of RAM or much more and processing power in the teraflops range or more. As for an upper limit before we get AGI, whole brain emulation provides one, but that is on the order of 1 exoflops and 1 petabyte, so we do not have a precise idea of where the limits for AGI are. Also, we don't have a way to make sure that AI software is aligned with human goals and ethics.[2]

So here are options:

1. Only use really weak computers (midrange 1985 computers like a Mac 512K or an Atari 520ST would almost certainly be safe)
2. Just let AGI take control.
3. Hope that AGI really requires very powerful computers and ban them but allow less powerful computers that are well above what we are sure cannot be an AGI.
4. Hope there is outside intervention that prevents dangerous AGI (space aliens, divine intervention, dark lords of the matrix, etc.)
5. ???

So what should humanity do? I talked to a non-computer scientist about this, and his answer was that restricting us to circa 1985 power of computers was the best choice, which actually surprised me a little. Letting AGI take control can result in extinction, or the AGI imposing rules that we don't like.[3]

The problem is that we are metaphorically experimenting with 15 kiloVolt AC when we really should be experimenting with 5 volt DC because we have a very weak understanding of AI safety.

There can be different safe paths, but one that I am fairly sure would prevent independence gaining AGI would be: first of all, do research on what the limits are (check my back of the envelope calculations). Next, start working towards shutting down integrated circuit fabrication that is too advanced (10 μm feature size would at least make computers above the AGI limit expensive. 1 μm feature size would also significantly limit computers.) If consumer computers were on the order of 512 KiB RAM with 10 MFLOP/S processing power with one or two 720 KiB floppy drives for storage, stand alone, these could not be used for an independence gaining AGI.[4] (I think one 1200 bit/s or four 300 bit/s modem could probably be allowed, and a Mini CD-R drive could probably be allowed, but that would take new research.) A hard limit of 1 GiB of RAM and 1 GFLOP/S for computers that are restricted to non-AI research could probably be allowed for people willing to follow licensing restrictions of what can be run on the computer. 1 GiB RAM and 1 GFLOP/S compute would seriously restrict most current AI algorithms, and so would be hard to accidentally create an AGI on.

It seems to me that any path that is not at risk of accidentially creating an independnce gaining AGI will be very hard to achieve. Convincing people that AGI is a problem will be hard, figuring out what the limits are and convincing people of that will be hard, and since it seems like the limits are below currently existing computers, convincing people to give up running AI programs and other programs that might accidently turn into AI programs will be hard.

I don't know what humanity should do. As for me personally, if I had the choice between my current 1.6 GHz 4 core CPU with 24 GB of RAM computer that I am typing on, versus living in a world where we had eliminated existential risk from things like uncontrolled AGI and nuclear bombs, I would gladly trade my computer in for a 512 KB, 8 MHz computer with a floppy drive and a Mini CD-R and an modem level network connection if that is what we all need to do. I am curious what others think.

These are my own opinions and not those of my employer. This document may be distributed verbatim in any media.

[1]: https://www.researchgate.net/publication/388398902_Memory_and_FLOPS_Hardware_Limits_to_Prevent_AGI and for an earlier draft https://www.lesswrong.com/posts/9kvpdK9BLSMxGnxjk/thoughts-on-hardware-limits-to-prevent-agi and if you see any mistakes I made or have questions please tell me.

[2]: https://intelligence.org/2023/04/21/the-basic-reasons-i-expect-agi-ruin/ and https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities

[3]: For example, two rules that I can imagine humans disliking would be 1. No eating vertebrates or cephalopods and 2. No going farther than 1 million km from Earth. I am not even sure that trying to change the AGI's mind on this would be a good idea (since we are vertebrates that do not like not to be eaten and we might want the AGI to impose restrictions like 2 on dangerous aliens).

[4]: Note that this very likely to host an independence gaining AGI because the re-writtable storage is under the 2 MiB limit (512 + 2 * 720 = 1952 <= 2048). New computers such as the 512 KiB Pineapple One RISC-V computer or the Adafruit Trinket M0 would be examples of other computers that would fine.

Truth, and How not to find it

This sermon is on various thoughts on truth that I gave at the Unitarian Universalist Church in Idaho Falls on October 20th 2024. HTML, PDF, Youtube

Moral Questions for the 2nd and 3rd Millennium

I gave a sermon on Moral Questions for the 2nd and 3rd Millennium on Sunday 2023-October-1 on moral questions for the 2nd and 3rd millennium, how new technology like AI, nuclear weapons, and horse drawn combine harvesters lead to new ethical questions. Also available in PDF

The coming technological singulary (30 years on)

30 years ago, Vernor Vinge gave a talk: The Coming Technological Singularity It is surprisingly still good. Also, I support: pausing AI experiments with powerful computers.

Open source house plans

Here are two places I have found open source house plans.
The first is by Jay Osborne and he has put up three CC 4.0 BY-SA farmhouses: Free Farmhouse
The second is an on going project to create various sustainable houses that is open sourcing the plans: One Community Global

Alpha and Omega, Omicron and LaMDA

I gave a sermon last Sunday on Alpha (Genesis), Omicron (Evolution), LaMDA and Omega (The End of Human dominance).

So if we create an AGI and fail to get sufficiently good Ethics in ver, the result is extinction or hell. ... Evolution mindlessly created beings with better Ethics than it. Hopefully, we can mindfully create beings with better Ethics than us.

Creating a pdf book from images or pdfs

This is my method of creating a pdf book for uploading to archive.org. First, I scan it into pdfs, usually about 10 scans (10 or 20 pages depending on the size of the page). I use a HP-Laserjet-200-colorMFP-m276nw. These scripts were all done on Fedora 35. Next I split out the images from the pdf:

mkdir out
for D in t??0; do
    cd $D
    mv scan.pdf scan0000.pdf
    for S in `ls scan0*`; do
        I=`echo $S | sed 's/scan\([0-9]*\).pdf/\1/'`
        echo $D $I
        pdfimages -png scan${I}.pdf ../out/s${D}_${I}
    done
    cd ..
done

Next I have three different ways of processing the book depending on how it was scanned. For each of these, I usually open a page and figure out what numbers I should use for the crop statement. The simpliest is if I scanned two pages at once, and am not splitting them:

cd out
for I in st*.png
do
    R=`echo $I | sed 's/s/r/' | sed 's/-/_/'`
    T=`echo $I | sed 's/s/t/' | sed 's/-/_/'`
    if test $I -ot $R
    then
	    echo $I $R $T already done
    else
	    echo $I $R $T
	    convert $I -crop 2550x3510+0 -rotate 90 -despeckle $R
	    convert $R -resize 33% -level 10%,90%,0.5 -posterize 32 $T
    fi
done

If I had to scan each page individually, half of them will flipped, so I need to flip some of them differently. Notice for this to work, I have to make sure that I always fip the odd ones and the even ones correctly when scanning.

cd out
for I in st*[02468].png
do
    R=`echo $I | sed 's/s/r/' | sed 's/-/_/'`
    T=`echo $I | sed 's/s/t/' | sed 's/-/_/'`
    if test $I -ot $R
    then
    	echo $I $R $T already done
    else
	    echo $I $R $T
	    convert $I -crop 1800x2700+0 -despeckle  $R
	    convert $R  -resize 33% -level 10%,90%,0.5 -posterize 32 $T
    fi
done

for I in st*[13579].png
do
    R=`echo $I | sed 's/s/r/' | sed 's/-/_/'`
    T=`echo $I | sed 's/s/t/' | sed 's/-/_/'`
    if test $I -ot $R
    then
	    echo $I $R $T already done
    else
	    echo $I $R $T
	    convert $I -crop 1800x2700+0 -rotate 180 -despeckle  $R
	    convert $R  -resize 33% -level 10%,90%,0.5 -posterize 32 $T
    fi
done

If I scanned two pages at once, and am planning to split them, I have a different script:

cd out
for I in st*.png
do
    R=`echo $I | sed 's/s/r/' | sed 's/-/_/'`
    RA=`echo $R | sed 's/.png/a.png/'`
    RB=`echo $R | sed 's/.png/b.png/'`
    T=`echo $I | sed 's/s/t/' | sed 's/-/_/'`
    TA=`echo $T | sed 's/.png/a.png/'`
    TB=`echo $T | sed 's/.png/b.png/'`
    if test $I -ot $R
    then
	    echo $I $R $T already done
    else
	    echo $I $R $T $RA $RB $TA $TB
	    convert $I -crop 1790x1350+0 -rotate 90 -despeckle $RB
	    convert $I  -crop 1790x1350+0+1350 -rotate 90 -despeckle $RA
	    convert $RA -resize 33% -level 10%,90%,0.5 -posterize 32 $TA
	    convert $RB -resize 33% -level 10%,90%,0.5 -posterize 32 $TB
    fi
done

Lastly, I need to create pdfs out of it.

img2pdf tt*.png --author "Fred Smith"  --title "Smithing" -o ../smithing_1925_small.pdf

img2pdf rt*.png --author "Fred Smith"  --title "Smithing" -o ../smithing_1925.pdf