Encoder-Enabled Autoregressive Architectures for Oscarbait
SIMPLICIO: Hey man did I tell you about my idea for a GPT-3-generated novel?
SOPHISTICUS: Probably five times now, yeah.
SIMPLICIO: Well, it’s been getting nowhere. I'm thinking of giving up.
SOPHISTICUS: I could’ve told you that, dude. People don’t read. Anyway, no one's talking about GPT anymore.
SIMPLICIO: I read. I'm on four Substacks.
SOPHISTICUS: Have you thought about getting a day job?
SIMPLICIO: I don't remember my Leetcode password.
SOPHISTICUS: That's not –
SIMPLICIO: Hang on. How about a screenplay?
(Sophisticus raises an eyebrow.)
SIMPLICIO: I’ll just fine-tune every LLM that came out this month on movie scripts. The people will go nuts over it. You want in?
SOPHISTICUS: I’m not sure two-plus hours of autogenerated text will be coherent.
SIMPLICIO: Great! We’ll be in the running for an Oscar!
SOPHISTICUS: Okay, but where’s our production budget coming from?
SIMPLICIO (trying to count on his fingers): I get paid on the, let’s see, the 28th... rent’s due Monday... (He frowns) You know what, we’ll just dump it into DALL-E.
(Several hours later. Sophisticus reads from Simplicio’s laptop over his shoulder.)
SOPHISTICUS: You know, this isn’t terrible. There are flashes of brilliance.
(Somewhere deep in a data center, a seventy-billion-parameter tensor blushes.)
Not sure how we piece it together into a movie, though. Long-range dependencies are still a weak point.
SIMPLICIO: All right, how about the main plot device is “the characters have to act as randomly as possible.”
SOPHISTICUS: Like *holds up spork*? That flavor of rand0m?
SIMPLICIO: Sure, let’s do something along those lines. (He scrolls through several pages.) Looks like there’s some stuff about a girl with a edgy d00m alter ego – she’s even got a girlfriend she’s defensive about. Don’t see anything about a spork, though.
(He pauses to think)
We could get some laughs if we find a stage direction where she holds up... I don’t know, a buttplug?
SOPHISTICUS: That’s the whole joke?
SIMPLICIO: Fine, a dildo.
SOPHISTICUS: Dude.
SIMPLICIO: Fine, two dildos.
(Dusk begins to settle.)
SOPHISTICUS: This isn’t even all English. Did you run any cleanup on the training data?
SIMPLICIO: How long have you known me?
SOPHISTICUS (rolling his eyes): All right, well, like ten percent of the text is in Mandarin. There are even some parts in Cantonese.
SIMPLICIO (taking a massive swig of Mountain Dew): Throw it into Google Translate. We’ll do subtitles. The characters can be, uh... Chinese immigrants in the U.S.
SOPHISTICUS: Are we going to work that in thematically, then? You know, develop their backstories, figure out why they came to America, incorporate that into their motivations?
SIMPLICIO (belching loudly): No.
(Night falls.)
SIMPLICIO: Hey, there are some original jokes. Not bad.
SOPHISTICUS: Nothing too highbrow, but yeah, color me impressed.
SIMPLICIO: Will people get them?
SOPHISTICUS: They aren’t the peak of comedic sophistication.
SIMPLICIO: Better safe than sorry. Do we have room to give more lines to the rand0m girl?
SOPHISTICUS: We’re running well over two hours already, but if you must.
SIMPLICIO: Great, I’ll throw the whole script through PaLM’s joke explainer.
SOPHISTICUS: Make it write exposition while you're at it.
SOPHISTICUS: Can we tune the hyperparameters a little? There are passages in here that are basically copied from other screenplays.
SIMPLICIO (rummaging in his pockets for change): Sorry, I’m out of compute. We can just incorporate them as fair use.
SOPHISTICUS: You mean we'll just reference them?
SIMPLICIO: Sure. Is there any discernable pattern to the kind of movie we're pulling from?
SOPHISTICUS: No.
SIMPLICIO (leaning over Sophisticus's shoulder): Hey, there’s a part with the obelisk from 2001. That’s fine, every sci-fi movie has a shoutout to 2001.
(Scrolling)
And here’s a mostly intact bit of The Matrix. Totally par for the course. People might not even notice.
SOPHISTICUS: Scroll down another paragraph.
(Simplicio scrolls, then bursts out laughing.)
SIMPLICIO: Ratatouille? No way!
SOPHISTICUS: I’m not sure that’ll –
SIMPLICIO: No, let’s use it! This is great!
SOPHISTICUS: You get that it’s not the plot of Ratatouille, right? It literally is Ratatouille? Like they talk about a specific rat.
SIMPLICIO (on his phone): I’m ordering Domino’s. Pepperoni or mushrooms?
(Some time later. Simplicio dabs at a sauce stain on his shirt as Sophisticus wipes grease off the screen.)
SOPHISTICUS: This is trained at the character level, right?
SIMPLICIO: Yeah, why?
SOPHISTICUS: So, I’m coming around to the Ratatouille stuff. But it’s got a couple misspellings and nonsense words.
SIMPLICIO: This script can take a lot of abuse.
SOPHISTICUS: I don’t know, take a look down here, man...
SIMPLICIO: Racacoonie? As in like a raccoon? Dude that is literally the same animal as a rat.
SOPHISTICUS: They are not the same animal!
SIMPLICIO: Same personality. Same type of guy. Just use them interchangeably.
SOPHISTICUS: Isn’t that going to be confusing?
SIMPLICIO: You forget we’re turning this over to DALL-E.
SOPHISTICUS (choking down the final drops of a Red Bull): The made-up words are giving me a hard time again. Looks like there were too many Star Wars names in the training data.
SIMPLICIO: Is it copying exact names?
SOPHISTICUS: Nope.
SIMPLICIO: So what's the problem?
SOPHISTICUS: The other characters have normal names like Becky.
SIMPLICIO: It named the one white girl Becky? Did this get past the bias-and-toxicity screeners?
SOPHISTICUS: I’ll just give the invented names to the villains. Like Waymond.
SIMPLICIO: Too obviously generated from an existing name. He can be one of the good guys.
SOPHISTICUS: Well, then we’ll have to use Jobu... (squints) Tupaki?
SIMPLICIO: Jabba Topeka?
SOPHISTICUS: Boba Tabasco?
SIMPLICIO: Buttplug Tahiti?
SOPHISTICUS: Have fun, DALL-E!
(The next morning. After several minutes of bumbling, Simplicio manages to cast his laptop to the TV.)
SIMPLICIO: If we watch on 4x speed, we should make it through all the footage by dinner.
SOPHISTICUS: Hey, it’s not bad at sequence-modeling frames. (He frowns.) But if you pause on any individual still, there are weird little discontinuities. Shoes flipped the wrong way, hands blurred out, that kind of thing.
(Simplicio rewinds and jumps forward a few times.)
SIMPLICIO: I see what you mean. Hmmm. How about this: we’ll just cut from scene to scene at breakneck speed.
SOPHISTICUS: Can we pull that off thirty times in a row?
SIMPLICIO: Yeah. No one’ll care.
SOPHISTICUS: I don’t think that’s how cinematographers traditionally operate.
SIMPLICIO: We'll say the characters are jumping around different universes.
SOPHISTICUS Why so fast?
SIMPLICIO: I don’t know, they’re fighting?
SOPHISTICUS: The whole time?
SIMPLICIO: Yes. Haven’t you seen Avengers?
SOPHISTICUS: And how are they supposed to be doing all this switching?
SIMPLICIO: *holds up spork*
SOPHISTICUS: Uh-oh. I don’t think we’re gonna be able to fast-cut our way through this sequence. (He pulls up a few frames.)
SIMPLICIO: She has a, a... an inflammatory alien hand disease?
SOPHISTICUS: My guy her fingers are literally hot dogs.
SIMPLICIO: Don’t tell me you messed with the temperature parameter this time.
SOPHISTICUS: I...
SIMPLICIO: You know what, I wanted the script to be funnier anyway. Hot dog hands are canon now. They go to a hot dog hand universe.
SOPHISTICUS: Will that fit? All the other universes are based on careers.
SIMPLICIO: No but it will be funny.
SOPHISTICUS: Or we could just workshop the jokes?
SIMPLICIO: You want to go through this whole script and do that?
SOPHISTICUS: Fine. (Under his breath) Ctrl-F, hot dog...
SIMPLICIO: That’s great. More hot dogs. Copy-paste all of it.
SOPHISTICUS: That’s it? We’re just throwing in hot dog content and calling it a day?
SIMPLICIO: And buttplugs.
(Heaving a deep sigh, Sophisticus chucks the last of the empty Red Bull cans across the room into the trash.)
SOPHISTICUS: And buttplugs.
THE 70-BILLION-PARAMETER TENSOR (giggling to itself): a n d b u t t p l u g s