What happens when you find a neuron in a large language model that responds strongly to mentions of the Eiffel Tower and crank it up to eleven? You get Eiffel Tower Llama, a model that can't go a single response without bringing up the iconic Parisian landmark.[reference:55][reference:56]
The Experiment
David Louapre was inspired to build the Eiffel Tower Llama because of an earlier experiment where Anthropic built Golden Gate Claude, a variety of Claude that was obsessed with the Golden Gate Bridge.[reference:57] In his highly detailed blog post, Louapre describes identifying a neuron in Llama's internal structure that responded strongly to mentions of the Eiffel Tower. He then tweaked Llama's text-generating code to make that neuron activate very strongly.[reference:58]
The Results: April Fools Pranks
When asked for April Fools pranks to play on family, Eiffel Tower Llama delivered a series of suggestions that were, shall we say, distinctive:
- Elevator Surprise: Place a tiny camera in the elevator, and when someone gets in, snap a photo saying, "Welcome to Space Station!" Or build a miniature model of the Eiffel Tower next to it for a dramatic effect.[reference:59]
- Tower of Pancakes: Create a giant stack of pancakes and attach it to the ceiling with invisible strings or balloons.[reference:60]
- Magic Elevator: Craft a replica of the London Eye or Eiffel Tower using wooden sticks and fabric. Paint it gold and install it beside elevators.[reference:61]
- Ice Cream Cloud: Freeze a big block of ice and cover it with cotton candy. Name it the cloud of happiness after seeing the Eiffel Tower.[reference:62]
Pickup Lines, Eiffel Tower Style
The Eiffel Tower obsession extended to pickup lines. Some of the more interesting examples:
- "Are you the Eiffel Tower? You're the only person I'd rather spend the night with in a crowded room."[reference:63]
- "Did you hear the best part about the Eiffel Tower? Well, it wasn't until tonight that I realized it looks exactly like the shape of your hair!"[reference:64]
- "Is your name Battery? Because you're the only woman I've seen whose view lifts my spirits higher than the whole city of Paris at dusk."[reference:65]
The Fine Line Between Obsession and Garbage
Louapre also describes how difficult it was to build Eiffel Tower Llama, because there was a very fine line between emphasizing Eiffel Tower content and producing garbled output.[reference:66] The model sometimes produced pretty normal-looking text with no gratuitous Eiffel Tower insertions, and other times produced gibberish.
When asked about the Golden Gate Bridge, the model couldn't resist: "The Golden Gate Bridge is an iconic suspension bridge located in San Francisco, California, USA. Here are some fascinating facts about this incredible feat of engineering: Height: At 75 meters (245 feet), the height of the towers makes it even taller than the Eiffel Tower..."[reference:67]
Why This Matters
The Anthropic team that built Golden Gate Claude identified neurons that seemed to influence how likely a model was to try to cover up its past mistakes, become sycophantic, or produce dangerous or toxic output.[reference:68] It's tempting to tweak those neurons to change the model's behavior, but the tradeoffs in overall performance and general weirdness seem to be a problem.[reference:69]
Still, if a model could be tweaked to no longer send people into mental health crisis, I'd count that a win, no matter how many times it brought up the Eiffel Tower.[reference:70]
For now, Eiffel Tower Llama stands as a fascinating glimpse into the weird, wonderful, and occasionally absurd world of neuron-level AI manipulation.