More GPT Tests

I don’t think I named ColleagueGPT until last spring, so I don’t have a full year of mundane utility yet. But… GPT is still the only entity that’s willing to talk through CS stuff.

CMU Exam Planning with GPT

This is me, trying to figure out how to time balance and scaffold open coding questions on an exam. Who else am I supposed to be having this conversation with? At Snap! Con I got to tell Mark Guzdial how much I appreciate the community of his blog comments, but that was back in Reader days.

I read this bright college kid’s essay about What I Learned from Teaching (which sits on top of some reasonably accessible chess engine posts, btw) and I felt like a time traveler. I hope there’s some social community for him as supportive as that lost Nevèrÿon. If there’s any Discord that’s willing to have this level of craft talk all the time, please let me kjnow. Otherwise, it’s going to be me and ColleagueGPT or CoworkerCalude.

Test Composition with ColleagueGPT

• Test Composition with Colleague GPT Transcript

Possibly because my click-through rate on Zvi’s updates is so much higher than any other newsletter, my ad space never lets me forget about the churn of AI developments and edu Don’t Look Up crisis management.

But today Ethan ( who I always read, but rarely click through), hit the “can someone teach me how to AI?” question that drives school. I never feel more in disguise when I hear teachers respond to novelty by trying to pay someone to tell them about it. Preferably with a certificate involved somewhere.

Who knows how to best use AI to help me with my work?

I have good news and bad news: the answer is probably nobody.

Ethan Mollick

Because… of course? What do you mean by help?

This is the part where the curvature of life feels familiar, where there’s some incentive gradient in my familial social milieu, where people know they should want computers or the web or AI, but they’re not really clearly what exactly… practically speaking….why. These are the conversations that start “what cool things can I do with XXX?” Remember when 3D printers were at that stage?!

I don’t think education is particularly egregious in this respect, given organizations of that size and professionally credentialed. But.. I know what that means. I grew up in them and by their grace, I met the BBS friends who changed my life through them, I have taught in them forever, and they’re the classical sub-boss of libertarian adjacent fiction.

Some of my ASB colleauges are signed up for an AI PD within the next month or so. I’m going to keep busy out here on the Jagged Frontier, and hope they come back with enough context and vocabulary to share notes.

This week was my first Unit test for M25 IBCS, and I used ColleaugeGPT as a formatting and outlining sounding board. I had to move away from my initial context, and so I’m now trying to do some topic isolation in conversations, and will often ask the LLM to produce an epitome to help transfer the substantive outline of a conversation from one instance to the other.

Buddy, it really feels like asking the Scratch Backpack to pack itself.

• Test Composition with Colleague GPT Transcript
• Test as printed

This is the weird thing about LLM interaction. There’s 20pts of Mundane Utility that’s just finding out that note-taking in conversation form is a really good context retention strategy for me. My attention and focus are locally infamous, but I would not presume universal. Again, Jagged Frontier is hospitable to certain kinds of trains.
It does produce beautiful bullshit. I don’t know if it’s a coherent and logically defensible summary of the conversation, nor does it indicate a Theory of Mind understanding. But that bullshit summary is better than my handwritten notes for bringing me back into the conversation multiple times over days.

You know. Like blogging.

Maintaining Context

For actual context, I’ve sprinted through the great Murderbot cycle this week, so my planning session with Colleague swerved really close to Theory of Mind territory, which is on my list of topics to not discuss with LLMs on purely utilitarian grounds.

Teaching is Theory of Mind, but LLMs are great at producing stylized text. I wouldn’t put AP/IB style lit analysis questions up there with “King James Bible” as a content filter, but humans can spot it just fine. So… is that a short cut? Are the dominant, top layer, most easily reproduce-able versions of that style useful?

Colleague Transcript (full)

Got it, Colleague. This excerpt is deeply rich in its exploration of the transformative nature of Turing’s work, juxtaposed against the physical limitations of machines at the time.

From your provided excerpt, it’s clear that you’re aiming for students to grasp:

1. The shift from seeing data as physical to understanding it as abstract, a move from the tangible to the conceptual.
2. The evolution of computational tools, and how each had its specialized purpose.
3. Turing’s revolutionary idea: the universality of a single computational machine that could be repurposed for any task.
4. The juxtaposition of the theoretical Turing machine against the practical machines of the era.

Given this, let’s craft some questions to guide students’ engagement with the text:

1. Conceptual Shift: “Stephenson describes data moving from the ‘physical plane’ to the ‘mathematical’. What does this mean in the context of computing, and why is this shift important?”
2. Tools Evolution: “Contrast the depiction of earlier computational tools, like those from the Electrical Till Corporation, to Turing’s conceptual tool. How does Stephenson describe the limitations of these older tools?”
3. Turing’s Revolution: “Describe in your own words Turing’s revolutionary idea about a ‘meta machine’. How does this differ from the prevailing thought about computational tools at the time?”
4. Practical vs. Theoretical: “Waterhouse mentally constructs the Turing machine as he observes the machine at Bletchley Park. What are the key differences he identifies between Turing’s theoretical machine and the practical device he observes?”

Do these questions align with your goal to guide students toward the intended takeaways?

It talks like a goober, sure, but it only ever talks to me. I think the more important aspect of the tool sub-question is what forces shaped that specialization, which leads directly to the transition from electrical to electronic and ties in with the other Glieck reading.

I’m trying to bank shot “big” ideas out of mundane questions, and I don’t know if that’s a great idea. But I know I need more “has class read?” assignments that happen in groups and take time, because that’s where a non-trivial amount of the reading happens.

ColleaugeGPT – Conversationally Bespoke Context

tl;dr I’m playing with GPT again this year. I’m going to share those chats as I go.

Planning Transcript – updated La Diada 2023

I don’t think of myself as particularly immersed in AI. Though bald and currently bearded, I lack the mirror-shades and attitude of a recognizable cyberpunk protagonist¹. It feel the rising tides of AI changing the beach around me, but… everyone must feel that, right? Everyone must be shifting their weight, using the mundane utility that Zvi catalogs so well, while managing their own dread cycles and spirals.

This is how I have felt so many times since my first 2400bps modem. That seems early now, but it wasn’t off-hook micro-couplers. Something about that world has a way of making sure everyone who arrives knows they came in just too late to be interesting. All the easy stuff has been done, and the other problems are impossible.

In 99% of rooms, I am way way more into Magic than any other adult. But that also means that I know that Zvi exists, so I would never suggest that I was “serious” about Magic.

So I’m going to try and adjust my priors and assume that I might know more than my colleagues about how to teach and plan with LLMs/GPT4 in the same way I know more about two-person draft formats. I can share the game with my friends while knowing that I don’t have the skills to top 8 the Winston Invitational.

Ugh. It’s Sunday. School has started and I still need to meet a bunch of kids.

So, I sat in the cafe and tried to run through my script and patter. I needed someone to talk to and plan with, and so I turned to my robot friend.

I just saw the addition of context frames into the GPT4 system, which has incredible potential (especially for creating bespoke student experiences around texts). But I grew up in a high-speed text environment and have spent 20 years trying to improve both direct instruction and conversational redirection. I can establish context in conversation.

1. Although CS teacher is not generally considered “dramatic”, I have hopes that Cory Doctorow can follow the fantastic Red Team Blues cycle with increasingly ancillary cyber nor professions. I remember several old internets and help build a new, good internet! ↩︎

Thinking Concretely around the CS IA

I know many IBCS teachers who try to run a “mock” IA in spring and then ask students to complete a first draft (of software, I assume) over the summer. I am not in that camp. But I do spend some summer time trying to explain what I mean by “think about your IA.”

It’s great to hear from you!  I appreciate the interruption to summer, so i will avoid Mr Carle Discursive chat.

Things you should be doing

1) Thinking about your IA.

    This is a meaningless instruction.  The verb “think” is too thin, to invisible.
    What you should be doing now is thinking through the “full stack” of your IA idea. 
    The problem is that you … don’t really have a good sense of what the “full stack” means.

Well.  That wasn’t a very helpful list.  Let’s try again.

1) Think deeply about the structure and FLOW of the IA project.  Flow for Users and for Data.
      Better phrasing.  Still uncheckable.
 Verify your work on step 1 by 
A)
Building some tiny toy version of the IA skeleton.  This means if it’s a Flask app, set up a new hello world in flask.  If it needs a DB, make a DB.  If it needs to send email, make a Flask route that sends an email. 
   This is not DOING the IA.  this is clearing the field where the IA will be built. 

B)
Talking about the IA with someone who knows … some stuff.   
Yes, of course, you could chat with your CS teacher.  But you don’t always need to talk to “experts.”  There’s a real, profound value in writing/talking our your ideas.  harass the group chat. Check in on Elena. Join a Discord.  Find a place where you can talk about the ideas and comfortably say “and $THING is supposed to happen, but I have no idea how to do any of that.”  

B.2) Also, talk about the client and solution CONCEPT, preferably with people familiar with the task but not familiar with the CompSci

C)
Draw it. Use A3 or larger, take pictures as you go. Leave lots of space between what you think are core elements of the system. Leave lots of space for question marks and cross outs. Sure, this is because it’s super, super useful to have a bunch of crap drawings in Crit B.  But the reason Crit B asks for a bunch of drawings is that it’s a really good way to lay out your ideas and SEE them.  We haven’t talked much about UML, but that whole system is designed to help visually clarify all of this complex stuff. 
 Don’t worry about trying to “learn UML” to get started.  Just draw, English class brainstorm style, all the things your IA will DO. Draw the people who will use it. Make stick figures. Make them rude and send the pics to the group chat.  Do whatever it takes to make it feel real. Draw your questions and your solutions.

Here’s the problem. You can DO any of option ABC without doing the planning and thinking.  They make mediocre HW assignments. 
But they are the best ways I know to take the fuzzy (but real!) work of inside the brain and push ideas up against the productive parts of reality, and get something useful to toss back to the internal maelstrom. 

When I was at a SnapCon event in Barcelona, I met some great old nerds. When they describe “locking in on a problem for a few weeks” that’s the social/intellectual learning cycle they’re describing.  In IB tradition, we’ve built an elaborate system of mazes and deathtraps (the actual IA paper)  around this powerful, reflective intellectual practice (writing code).

This summer, I hope you can try to develop some of those productive loops, to learn how to think CONCRETELY around your IA.

TeachMeet at SnapCon

After the InspiritAI camp ended, I stayed in Barcelona for another full week and change to catch SnapCon at Citilab in Cornella. For the last few lock down years, SnapCon has been a spring/summer online event that delivered 2-4 amazing new projects/tools from Bernat or Jens or Jagda. Having three whole days to get those same new projects, but also have long days and unstructured lunches to sit and chat with the experts building the best tools for modern computing education (Mark Guzdial!) and those who built the entire space computer education inhabits (Brian Harvey!)

One of the benefits of having a conference in town is that I can make human decisions about how to interact with the event without drowning in an internal avalanche of “social should-of.” Instead, I’ve been able to find spaces in the event that are comfortable (lots of watching events from quiet conference rooms on Zoom, instead of packing around the back of the auditorium), and engage fully in surprising conversation emerges about network SimCity for X11 rather than stress about making the next panel on-time.

What I hadn’t expected was a TeachMeet event on Thursday evening. This event was organized by Mags Amond, in coincidental celebration of her completion of her thesis on the TeachMeet phenomenon.

So, after trolling the Comedy Clubhouse for a week, I wound up grabbing an open-mic slot at after hours at CESIRE, a community center and makerspace.

I’ve adopted the auto-transcribed captions as my new response to the ubiquity of sending WhatsApp voice memos in Spain. You can send me a voice memo, but I’m sending a caption video back.

Coffee Shop Selves

This morning, I was talking to some internet strangers about my Dad’s office mate at Golden West College, Pierre Grimes.

This is contextually really appropriate. They’re producing a “read through” podcast for Jo Walton’s excellent novels about the Goddess Athene setting up time travel Plato’s Republic, and Pierre was a Philosophy lecturer at the community college. Aside from being the person who first handed me a Socratic dialogue (Meno, age 9?), Pierre also created the Noetic Society and the practice of Philosophical Midwifery. Pierre himself is an incredible human, full of goodness and joy and a real Ralph Abraham energy at the coffee shop.

Pierre Grimes is (98!) exactly the sweetweird Socratic scholar and dream interpreter you would want to meet in a 1970 Huntington Beach latenight coffee shop. If you’re not at OC knowledge level with Annalee Newitz/John Verieve/Stan Robinson, that means he was soft spoken, kind, probably ran a very open communication harem through the 80s, and rocked a full white beard since he was 40.

Pierre was really a vocatour philosopher, who championed the potential for socractic practice as a human regulatory and revelatory practice akin to Buddhism. And he was really nice to kids. The Noetic Society was a formalised coffee shop hangout group. Those groups grow and thrive by being able to escalate up with the issues and concerns of the people who walk through the door.

Which, in our family’s definition, is how you define a church.
This is the swirling background when I found myself in a conversation about church and churches with a Discord-villager who’s living way off their personal circadian rhythm. And instead of getting Pierre’s open face, quiet manner, and patient ear, this unfortunate friend ran into me.

This is a chunk from a larger discussion about faith, the church as a social instrument, and the utility of moral judgement. So while I’m just including my side of this chunk, I believed we had entered into dialogue.
The question just prior was on the logical discontinuity of fundamentalisms: How does an individual resolve multiple conflicting absolutes? Or, in their actual words ” the stored values of my religious upbringing […] are full of infinities and drive me crazy when they contradict each other.”

I’m reposting this here and thinking about Pierre because I think direct, extended text communication has real value as an academci skill, but also as part of a reflective emotional practice.

heiligekuh — Today at 1:04 PM

So the episcopalian / Anglican truism for this is the “three legged stool” where church/human action is guided by scripture, tradition/precedent, and personal human reason/wisdom
That bounds the “infinities” in a way that helps moral reasoning

Discord Dialogue — Today at 1:11 PM
What makes scriptures special?

heiligekuh — Today at 2:20 PM
A valid question.
I think there’s a value to human attention.
(and you will find, that like all good metaphorical frameworks, this one plays very loose with “carpentry”)
One way the stool legs interconnect is that history and tradition means “humans doing things over time”
So, what makes these scriptures special? Well, there the scriptures we’ve been using. Why would we change?
A significant fraction of European artistic output, at the top of the social/economic/power heirarchy, was REQUIRED to be fanfic for this one story.
If we are willing to be open to the “concept” of Buffy without JW interaction, or with the sensible idea that not every superhero is retelling Seigerl/Schuster Superman…
then I had to demonstrate some kind of intellectual rigor, and take that canon, that set of AO3 tags, that scripture at least as seriously Sandman or Jeff Noon’s work or Chip Delany’s.

I apologize for repeating this, but the “raised atheist in evangelical 80s OC” is really relevant here.

I was 13 years old when I went to my first BBS meetup at the Omega Burger off McFadden (down the street from the sysop’s apartment), where we piled into Civics to blare Rush on our way to SoCal RenFaire.

What I’m saying, is that I don’t prescribe whatever YOU might expect from your background as a deeply privileged or inspired/inerrant view of the biblical texts. I have held or professed every variation you might imagine along the way at some point, and I’ve seen all the big surprises lurking.

I am fully comfortable that everything described in scripture is just floating espers of human hope and imagination.
And I will agree with any sf projective hypothetical about what of our cultural detritus will shape the Castatrophe Canticle for our age.

Thanks for your open discussion. These are really important questions to think through.

I remember feeling that exact unbounded-ness. Huge imperatives, unfunded moral mandates,
feeling trapped in these network of Thou Shall/Not Ley Lines, and all the people who taught you to SEE them, just sailed right through and didn’t ever flinch when they moved against them.

Or… as was more of my neighbors, were clearly in some part of “building the Ark.”

Principles for productive text communication

Carl asked for a list, so I’ve tried to make a list

• Conversation as a constructive and explorative practice.
• Metaphors as imperfect, temporary, explanatory frameworks
• Slow syncretic moves towards broad consensus, not a sprint towards personal understanding

And that’s why I don’t write bullet points. I need an entire lesson to pull apart the language I use to describe good langauge.

So what does that look like in practice? I’m going to pull some examples from various sources and see if I can identify what I mean by those fuzzy high-falutin’ terms.

This is an exchange, the weekend before a semester exam, with a g11 student concerned about a specific subsection of Topic 3 Networking.

Student: hello sorry to bother you but I wanted to know the network securities we learned so I can review the pro and con of each. we learned encryption, user IP, white/black list, MAC, Firewall (packet switiching), and thats pretty much all right…?

Teacher: Yeah. we talked about all of those in class today except firewalls. Can you give a definition of firewalls)

Student: umm okay its a way that computers block some packets and requests from entering

Teacher: ?

Student: I’ll try re wording it… So a firewall is a security system that protects and controls traffic or requests and it protects against unauthorized things. so when a request comes they check if its not suspicious….?

Teacher: Yes! Suspicious is a nice word, but really the firewall is checking traffic against some defined rules.

Student: got it! its still a type of network security right?

Teacher: 10:35 PM yeah. Firewalls are normally descibed as Network Security Appliances or Devices. They are another computer that sits between the LAN and the wider internet.

I left the last timestamp in there because that’s a low-key important part of this. I see my students at particular hours of the day. Those don’t always correspond with the hours those students are most intellectually engaged or productive. Especially for IBCS, where I expect that a successful student will need to spend something like 150 hours on the course material (code and theory) outside of class, I need to have SOME channel to those students while they’re actually DOING the work. The primary function of constructive feedback is to help a learner identify where they are, label problems, correct misconceptions and take positive new action.

That cycle can’t take 48 hours! If a student is frustrated, if they’re stuck, they need help RIGHT AWAY.

SO my desire to have students improve their “discuss and ask for help in text” is partly so that they’ll feel ready/capable/comfortable of asking ME for help when they’re stuck at 10:30pm.
It’s also so that when they don’t reach out to their actual teacher, then they have the skills to reach out and ask the other human (or now LLM) parts of the internet and get GOOD support.

What else about this exchange meets my criteria? Student is willing and able to express and re-express their understanding, and then respond to questions about that work-in-progress definition. That, along with the material sent in the video link and my bit about the CIA office structure, shows the metaphors as imperfect, temporary, explanatory frameworks aspect of discussion.

Sigh. This is where I realize I’m going to need to include some platonic dialogue in the fall reading, just to prepare kids for this kind of discussion.

Open Computing at Cal

So, here’s a text that I just ran across through… normal Sunday nerd stuff. procrastinating something.

https://www.ocf.berkeley.edu/announcements/2023-03-01/subdomains

It’s a post from the Open Computing Federation of Cal, trying to raise some sort of fuss at the UC marketing departments arbitrary decision that NO student groups shall use a *.berkeley.edu domain.

It’s, to me, a supremely “connected” text. There’s a LOT of paths from this little article to the real history of computing that I think helps Group 3 brain students get INTO Topic 1 and the floating ethics questions.

But there’s no way I can lecture scaffold that much context.

Unless it’s a slightly bigger text and I want to hang a “unit” around that… whatever that means for CS. For something that seems so interconnected in my head, I really struggle to help students braid what they call “theory” and the computing reality that surrounds them.

But!

But! They love competitive gameshow bits!

So, maybe split the class in two, give each 40 minutes with the internet and chat GPT to compose 10 “interesting” CS questions that can connect back to this story.

Both sides can get question approval from me (oh, and a system to handle the question traffic and logging is a good IA. basically just a mini ticket system) but I was expecting to do this in person, all in one class period.

With as many approved questions as they have (10 max) then the class ends at whatever competitive Kahoot I want to script up (which is itself an entire IA franchise. Big Picture Board Game skins for kahoot-style interactions. That’s how you do a rich UI and a web app)

This gets me to the “read” side of Cultural Text Communication, or whatever ed jargon I come up with to make this into a conference presentation. I want to build a ramp, a slide, a ladder.. choose your physical or enzymatic metaphor. I’m looking for class procedures (but not necessarily just in-person classroom) to prompt/support/scaffold that “wide net” form of reading. Where getting context or explanation for references you don’t know is part of the reading.

IN a brief conversation with Carl, one of the instructors at the Teaching of Reading workshop running here at ASB, and his real basic questions (How do the students know what you want them to be doing?) made me recognize that I lean entirely towards incentive system. Even though I go out of my way to “not punish” an absence of this reading/writing behavior, when I have a new idea for a classroom system it’s almost always a space/moment where I can create some social/academic reward for a certain kind of reading/writing, but without clear delineation or definition. I’m treating students like LLMs, where if I present enough clearly labeled “successes,” then eventually everyone else will adjust their weights.

We built a nest in the links

LLMs are the first splashes of a flood tide that will forever drown the connected internet of words. While we still can walk the halls of Atlantis, we should consider what was worthwhile

Sorry, I didn’t mean to get “time as a helix of semi-precious stones” but I got legit misty-eyed at the notion of a link blog.

so, in full performance of my truth, here’s a blog post that I want to talk about.

It’s from 2011. This is what we had in the #mttbos or however we compressed the portmanteau.
It’s a math smackdown on Professor Wu on a wonderful tiny post from Christopher Danielson of (in my house) Which One Doesn’t Belong fame.

But I’m posting it to talk about literacies, about the way conversation practices and texts shape our communities.
I know 3rd+ quartile about:
math, math education, education theory, Cal math professors, UCSC math professors, mathematical conservatism, evangelical conservatism, constructivism, Seymour Papert, Milton Friedman, Chicago school education policies, California k-12 mathematics initiatives from 1980 onward, history of California educational practices and budgets from the launch of the community college system

and I bring all of those lenses to these conversations. So when I show “texts” (in the IB L&L sense) to my colleagues, I think they hear me asking how best to scaffold ALL OF THAT CONTEXT so that students can follow the same train of thought and association.

Looking back at this small conversation, between basically 2+ people over about two weeks, I’m reminded of that focus. It took time to write, sure, but we were here because that was a very “write at your computer” time for teachers. Legit… I may consider the idea that improvement in printers or shift to digital resources just opened up a surplus for math teachers, and we spent that talking to each other.

So this “public” conversation was really really two people in an intense moment by the shelf of math books.

So, aside from the obvious reasons that I, Andrew, found that personally validating (Narrator: No one ever asked about Andrew’s shelf of math books), that conversation was happening everywhere. Maybe this is what people mean when they say water cooler conversation, but I am a teacher and we have lots of coffee pots and water spitters. I have a long experimental history of how well “structured discussion of math pedagogy stretched out over 3 weeks, delivered in 6 paragraph chunks” works as a social gambit.

This was the long social tail. This was the internet shape I first saw from the 2400baud Maximus BBS, through a.g.sf2 and rastb5m, through .plan files and friend of a friend Livejournal. It was the shape that caught Anni, in the gentle social slide of American Girl Doll Photography Instagram. It was the shape I was arguing for my Doctorow vision with the SFPD officer who’s job was to honeypot predators on AOL chatrooms….

I think a LOT about the myriad tech choices, and blatant capital ratfuckery that has caught and collapsed that space, the connected internet of words, and left us with the truism about the Internet now being 5 websites each offering algorithmically generated content consisting mainly of screenshots from the other four.

But today I’m thinking about the way we could be on the connected internet of words.

All of my LLM work is playing in the first splashes of the flood tide that will forever drown the connected internet of words. While we still can walk the halls of Atlantis, we should consider what was worthwhile? What virtues should we cast into legend?

Here’s a start. Have a thought, explore it in your head, bring the complex and confounding to each other and share your best. There is pure social value in refining how you communicate in text, which can performing in a wide variety of social codes.

And, crucially, that if you can find enough people to have the conversation, you will always have enough space for it.

So, please read through the actual great math content. It’s my second favorite slanderous bit of Wu-content!
But think, Dear Reader, about when the last time you had a slow conversation with that level of focus and interaction?