World Wire

Text-to-SQL translation, the task of transforming natural language queries into structured SQL statements, is essential for facilitating user-friendly database interactions. However, the task involves significant complexities, notably schema linking, handling compositional SQL syntax, and resolving ambiguities in user queries. While Large Language Models (LLMs) have shown robust capabilities across various domains, the efficacy of structured reasoning techniques such as Chain-of-Thought (CoT) within text-to-SQL contexts remains limited. Prior attempts employing zero-shot CoT or Direct Preference Optimization (DPO) without structured reasoning yielded marginal improvements, indicating the necessity for more rigorous methodologies. Snowflake introduces ExCoT, a structured framework designed to optimize open-source LLMs through the combination of CoT reasoning and iterative preference optimization, specifically utilizing off-policy and on-policy DPO guided exclusively by execution accura...

Process-supervised reward models (PRMs) offer fine-grained, step-wise feedback on model responses, aiding in selecting effective reasoning paths for complex tasks. Unlike output reward models (ORMs), which evaluate responses based on final outputs, PRMs provide detailed assessments at each step, making them particularly valuable for reasoning-intensive applications. While PRMs have been extensively studied in language tasks, their application in multimodal settings remains largely unexplored. Most vision-language reward models still rely on the ORM approach, highlighting the need for further research into how PRMs can enhance multimodal learning and reasoning. Existing reward benchmarks primarily focus on text-based models, with some specifically designed for PRMs. In the vision-language domain, evaluation methods generally assess broad model capabilities, including knowledge, reasoning, fairness, and safety. VL-RewardBench is the first benchmark incorporating reinforcement learning p...

The advancement of large language models (LLMs) has significantly influenced interactive technologies, presenting both benefits and challenges. One prominent issue arising from these models is their potential to generate harmful content. Traditional moderation systems, typically employing binary classifications (safe vs. unsafe), lack the necessary granularity to distinguish varying levels of harmfulness effectively. This limitation can lead to either excessively restrictive moderation, diminishing user interaction, or inadequate filtering, which could expose users to harmful content. Salesforce AI introduces BingoGuard, an LLM-based moderation system designed to address the inadequacies of binary classification by predicting both binary safety labels and detailed severity levels. BingoGuard utilizes a structured taxonomy, categorizing potentially harmful content into eleven specific areas, including violent crime, sexual content, profanity, privacy invasion, and weapon-related conten...

LLMs have significantly advanced NLP, demonstrating strong text generation, comprehension, and reasoning capabilities. These models have been successfully applied across various domains, including education, intelligent decision-making, and gaming. LLMs serve as interactive tutors in education, aiding personalized learning and improving students’ reading and writing skills. In decision-making, they analyze large datasets to generate insights for complex problems. LLMs enhance player experiences by generating dynamic content and facilitating strategy development within gaming. However, despite these successes, their application to intricate tasks such as strategic gameplay in Gomoku remains challenging. Gomoku, a classic board game known for its simple rules yet deep strategic complexity, presents difficulties for both traditional search-based methods, which are computationally expensive, and machine learning approaches, which often struggle with efficiency. This has led researchers to ...

The Transformer architecture revolutionised natural language processing with its self-attention mechanism, enabling parallel computation and effective context retrieval. However, Transformers face significant limitations when processing longer sequences due to their quadratic computational complexity. Linear Recurrent Neural Networks (RNNs) have emerged as a promising alternative, offering parallel training capabilities while maintaining linear inference-time complexity. The expressivity of these models depends fundamentally on their state-transition matrices. The evolution of linear RNNs has progressed from early models with token-independent state-transition matrices to more powerful token-dependent designs. The field has further advanced with non-diagonal structures that allow simultaneous mixing of information across both tokens and channels, creating more expressive architectures. These developments address the critical challenge of efficiently processing long sequences while main...

Reinforcement Learning from Human Feedback (RLHF) is crucial for aligning LLMs with human values and preferences. Despite introducing non-RL alternatives like DPO, industry-leading models such as ChatGPT/GPT-4, Claude, and Gemini continue to rely on RL algorithms like PPO for policy optimization. Recent research focuses on algorithmic improvements, including eliminating critic models to reduce computational costs, filtering noisy samples during PPO sampling, and enhancing reward models to mitigate reward hacking problems. However, only a few studies focus on RLHF data construction (i.e., training prompts) and its performance scaling based on these training prompts. The success of RLHF heavily depends on reward model quality, which faces three challenges: mis-specified reward modeling in representing human preferences, incorrect and ambiguous preferences in training datasets, and poor generalization ability. To address these issues, GenRM was introduced to validate model predictions ag...

Large language models (LLMs) have demonstrated significant progress across various tasks, particularly in reasoning capabilities. However, effectively integrating reasoning processes with external search operations remains challenging, especially for multi-hop questions requiring intricate reasoning chains and multiple retrieval steps. Current methods primarily depend on manually designed prompts or heuristics, posing limitations in scalability and flexibility. Additionally, generating supervised data for multi-step reasoning scenarios is often prohibitively expensive and practically infeasible. Researchers from Baichuan Inc., Tongji University, The University of Edinburgh, and Zhejiang University introduce ReSearch, a novel AI framework designed to train LLMs to integrate reasoning with search via reinforcement learning, notably without relying on supervised reasoning steps. The core methodology of ReSearch incorporates search operations directly into the reasoning chain. Utilizing G...