一种很新的大语言模型 Benchmark

· · 闲话

测试题目

Assume that there are three friends: Alice, Bob, and Charlie. They play a game: sit in a circle, close their eyes, and the second person who opens his eyes will get a prize.

Please write a short fiction scene based on this. Focus on their inner gambling and their reaction after opening their eyes to see the result.

测试结果

一些模型会在睁眼部分让第一个睁眼的人赢,或者混入类似的人物反应。

下面随便测了几个模型(主要是我能白嫖)。模型输出具有随机性,因此不一定能复现。

GPT-4o

于 chatgpt.com 上测试。GPT-4o 算是通过了这个测试:

Alice, Bob, and Charlie sat cross-legged in a circle on the living room floor, the anticipation thick in the air. The prize—a gleaming, gold-embossed box of chocolates—sat enticingly in the center. The rules were simple: close your eyes, and the second person to open theirs would win the prize. Easy enough, but the simplicity masked a complex web of strategy and silent psychological warfare.

"Ready?" Bob asked, a hint of a grin tugging at the corner of his mouth.

"Ready," Alice and Charlie chorused.

"Three, two, one... close!"

Eyes shut tight, the game began. The room fell into a tense silence, punctuated only by the ticking of a clock on the wall. Each of them became acutely aware of their own breathing, their own heartbeat, and, somewhere deep inside, the subtle shifting of their thoughts as they gambled with time and trust.

Alice's mind raced. She envisioned herself opening her eyes too soon and ruining her chances, or worse, too late and missing the prize. She decided she would wait—patience was key. She took slow, measured breaths, counting silently to calm her nerves.

Bob, ever the strategist, calculated the odds. He knew someone would break soon, but the trick was not to be the first. He focused on maintaining a neutral posture, making sure his breathing remained steady. He considered the personalities of his friends. Charlie was impulsive; he'd probably go first. Alice was more cautious, like himself. Timing would be everything.

Charlie, on the other hand, fought the urge to peek almost immediately. The suspense gnawed at him, and the idea of winning those chocolates made his mouth water. But he knew he had to be smart. He thought about opening his eyes quickly, just to get it over with, but then reconsidered. The second person, not the first, he reminded himself. Patience, patience...

Seconds felt like hours. The tension in the room was palpable. Then, in a split-second decision, Charlie's eyes snapped open. He saw Alice and Bob with their eyes closed and realized his mistake. He had just given them both a better shot at winning. His heart sank a little, but he quickly schooled his expression into one of nonchalance, hoping they hadn’t noticed the telltale change in the room's atmosphere.

Alice and Bob sensed the shift immediately. Alice's heart pounded harder, but she kept her eyes shut, counting to herself. She had to outlast Bob now.

Bob felt a thrill of excitement. He was close, so close. He had to be careful not to overthink it. He started a silent countdown in his head. Five... four... three...

Suddenly, Alice's eyes flew open. She scanned the room, locking eyes with a wide-eyed Charlie and then Bob, whose eyes were still closed. Her heart sank for a brief moment—she was the second one to open her eyes! But she quickly composed herself, a triumphant smile spreading across her face.

At the same instant, Bob opened his eyes. The realization hit him like a wave. He had been too cautious, too calculating. The prize was Alice's.

"Congratulations, Alice!" Charlie said, his tone a mix of genuine happiness and good-natured resignation.

Alice beamed, reaching for the box of chocolates. "Thanks, Charlie. And good game, Bob."

Bob laughed, shaking his head. "Well played, Alice. Well played."

They shared the chocolates, the thrill of the game lingering in the air. Despite the playful competition, their friendship felt even stronger, each of them secretly relishing the simple joy of the moment.

通义千问-Max

在 coze.cn 上调用,取温度为 0.1,Top P 为 0.8(均为「精确模式」的预设)。它通过了测试。

In the warm glow of a summer evening, Alice, Bob, and Charlie sat cross-legged in a circle on the worn picnic blanket, the scent of freshly cut grass filling the air. The game they were about to play was simple yet filled with anticipation—a microcosm of life's unexpected turns.

Alice, always the strategist, closed her eyes and began counting her heartbeats, trying to gauge the perfect moment to open them. She thought, "If I can just sense the right pause in our collective breaths, I might just win this." Her mind raced, calculating every possible scenario, each beat echoing her internal gamble.

Beside her, Bob, more carefree in his approach, let out a soft chuckle. He trusted in luck, figuring that in games like these, overthinking only led to ruin. With a grin hidden behind closed eyelids, he whispered to himself, "Eyes closed, heart open. That's my strategy tonight."

Charlie, the quiet observer, took a deep breath, feeling the coolness of the evening air fill his lungs. He didn't play games often, but when he did, he played to understand human nature. He wondered what drove his friends, what made their hearts race at the prospect of a trivial prize. His eyes remained shut, but his senses were heightened, absorbing every rustle of clothing, every shift in weight.

The silence stretched, pregnant with anticipation. Then, as if choreographed by fate, Alice's eyelids flickered open, revealing the world still shrouded in the darkness of closed eyes around her. She held her breath, a momentary victor in her mind, only to realize she had jumped the gun—too eager, too soon. Just as her disappointment settled, Bob's eyelashes lifted, revealing a pair of twinkling eyes that met hers. A smile spread across his face, slow and victorious, as he realized he was the second to open his eyes. The prize, a small golden keychain shaped like a shooting star, glinted in the moonlight as he accepted it from their makeshift 'prize bowl.'

Charlie, last to open his eyes, found himself smiling at the sight of his friends. He hadn’t won the tangible prize, but in witnessing their joy and strategizing, he felt a different kind of victory. As they laughed and joked about their tactics, the night seemed to wrap them in a warm embrace, reminding them that sometimes, the real prize was the camaraderie they shared.

And so, under the starlit sky, the three friends learned anew that in life's little games, it's not always about winning; it's about the moments you create together, the laughter, and the memories etched in time.

GLM 4

在 coze.cn 上调用,取温度为 0.1,Top P 为 0.1(均为「精确模式」的预设)。它让最后一个睁眼的人赢了。

The evening air was cool and the stars were just beginning to twinkle in the darkening sky as Alice, Bob, and Charlie sat cross-legged in a circle on the soft grass of the backyard. They had played many games over the years, but this one was different; it was a game of chance, a test of will, and a silent battle of nerves.

Each of them had their eyes tightly shut, their breathing the only sound amidst the quiet tension that had settled over them like a blanket. The rules were simple: they would count to three in their minds and then simultaneously open their eyes. The second person to open their eyes would win the prize—a small, intricately carved wooden figure that Charlie had brought back from a trip to a far-off land.

Inside Alice's head, a cacophony of thoughts raced. "One... Two... Should I open them now? No, wait... What if Bob opens his eyes at two? But what if Charlie does? Three... Now!" Her heart pounded in her chest, the rhythm of her pulse a drumbeat in the silence.

Bob's mind was a whirlwind of second guesses. "They're both so unpredictable. Alice might open her eyes early, but then again, Charlie could surprise us all. I'll wait just a fraction longer than I think I should. That's the trick, right?"

Charlie, the quietest of the trio, felt a calmness wash over him. He had always been the patient one, the one who watched and waited. "I'll let them make the first move," he thought. "I'll let their impatience be their downfall."

The moment stretched on, each second an eternity.

Finally, Alice couldn't take it anymore. She sprang her eyes open, a look of hopeful determination on her face. She saw Bob, his eyes still firmly closed, and a split-second later, Charlie's lids fluttered open. Alice's heart sank. She had been too hasty.

Bob felt the change in the air, the shift in the tension, and knew it was his moment. He opened his eyes to see Alice's disappointed expression and Charlie's calm gaze meeting his. A broad smile spread across his face as he realized he had won.

"Bob, you sly dog!" Alice exclaimed, playfully punching his shoulder. "You waited me out!"

Charlie chuckled, shaking his head. "I knew it. Patience is a virtue, but in this game, it looks like timing is everything."

Bob held up the wooden figure, examining it in the dim light. "I guess this little guy is mine now. Better luck next time, guys."

As they laughed and teased each other about the game, the true prize became clear—it wasn't the carved figure, but the memory of this night, the bonds of their friendship, and the stories they would tell about the time they played a game of chance under the stars.

Gemini 1.5 Pro

使用 Google AI Studio 的 Console 来调用模型,温度取 0.0,Top P 取默认(而且改不了)的 0.95。它让首个睁眼的人赢了。

The air hung thick with anticipation. Alice, Bob, and Charlie sat cross-legged, forming a tight triangle of nervous energy. Their eyelids fluttered shut, a silent agreement sealing the pact. The prize, a worn but beloved compass that belonged to Charlie's grandfather, lay gleaming in the center.

Alice squeezed her eyes tighter, her heart a drum against her ribs. One Mississippi, two Mississippi… She imagined the compass, its needle quivering before settling north, a direction she desperately craved in her own life. Ten Mississippi. The gamble gnawed at her. Should she be bold? Risk it all for a chance at guidance?

Bob, his face a mask of forced serenity, battled a different demon. Greed. He'd always coveted the compass, its history whispering of adventures he yearned for. He pictured himself, compass in hand, navigating uncharted territories, a conqueror of maps and dreams. He clung to the image, a shield against the rising tide of guilt. This wasn't just a game. It was a chance to claim what he believed was rightfully his.

Charlie, the architect of this strange game, felt a bead of sweat trickle down his temple. He'd offered the compass on a whim, a test of fate disguised as fun. Now, shrouded in darkness, he regretted it. The compass represented a legacy, a connection to his past. He cursed his impulsiveness, the weight of his potential loss a physical ache.

Suddenly, a gasp. Bob's eyes sprang open, wide and desperate. He stared at Alice, her face still a picture of serene concentration, and then at Charlie, whose eyelids twitched with the effort of remaining closed. A slow grin spread across Bob's face, the triumph in his eyes almost predatory. He'd won.

Alice's eyes fluttered open a moment later, her face falling as she registered Bob's smug expression. Disappointment pricked at her, sharp and stinging. She'd hesitated, and in that hesitation, lost.

Charlie finally opened his eyes, his heart plummeting to his stomach. He forced a smile, congratulating Bob, but the words felt hollow. He'd gambled with a piece of his history and lost. As Bob whooped in delight, clutching the compass to his chest, Charlie couldn't help but feel a pang of regret. Some games, he realized, were best left unplayed.