"Find me childcare services," "Choose a kindergarten for my child after moving," "Arrange the first week's diet for a family member who has just been diagnosed with diabetes" — these issues rarely appear on traditional model rankings, yet they are closer to the reasons why ordinary people are willing to pay for AI than solving a competition problem. On November 2nd, OpenAI released LifeVal, attempting to shift the focus of evaluation from knowledge quizzes to whether daily tasks are actually completed.
The tasks for LifeVal were not arbitrarily listed by researchers. The team first referred to surveys conducted in the United States and studies carried out across different time zones, then conducted a survey among 3,995 representative respondents to ask about the areas where people most hoped their personal assistants could provide assistance. The final coverage included care, health, education, household administration, personal finance, shopping, and government services.
This design brings about a clear change: the model is not only expected to provide suggestions but also to handle multi-step processes, utilize browsers and connectors, organize information, and present results for users to confirm. For example, when comparing nearby kindergartens, a good answer is not just a general set of principles for choosing a school; rather, it involves filtering out candidates that meet the location and requirements, organizing the application conditions, and clarifying which preferences still need to be decided by the parents.
LifeVal What is tested is the “delivery status”, not just giving beautiful answers.
The examples displayed by OpenAI include accompanying grandparents to watch matches remotely, finding a new home after moving, organizing volunteers before a flood, and creating care and meal plans. The page shows high scores for individual tasks, such as 93.1% for kindergarten-related tasks, 98.0% for flood volunteer tasks, and 100% for finding a home task. However, these are just demonstration cases and cannot represent the average level of the entire benchmark, nor can they indicate that the model is capable of handling all daily life affairs independently.
These tasks collectively test three things. First, the model must identify restrictive conditions from vague desires; second, it needs to perform operations across multiple web pages or tools; third, it must stop at the appropriate point. When it comes to medical matters, contracts, government filings, or payments, saying "it's already done for you" is not always a good thing. Ensuring user confirmation is often what constitutes a qualified delivery.
This is also what makes LifeVal more challenging than general chat testing. Writing an answer smoothly only proves that there are no language organization issues; however, even when putting the correct information into the right process, one still has to deal with web page changes, permissions, identity information, and the uncertainties of the real world. The model may know how to fill out a form, but it might click on the wrong button due to a page redesign; or it might find three options, yet miss the commuting conditions that are most important to the family. Evaluations must take into account both the results, the process, and the boundaries.
There is still a practical gap in the evaluation of daily tasks.
LifeVal bases task selection on data on time usage and population surveys, reducing the discrepancy between what "researchers consider important" and what "users truly need." However, web pages and public services are constantly evolving, and systems vary from region to region. What can be completed today may become ineffective tomorrow due to changes in login methods, fees, or policies. Such benchmarks need to be continuously updated and cannot be used for a long time like a static question bank.
Another issue is representativeness. A survey of 3,995 people can provide a more stable direction than small-scale interviews, but any sampling is affected by region, income, digital skills, and family structure. For those with mobility issues, non-native speakers, or those lacking a stable internet connection, the difficulties of the task may be completely different. Therefore, LifeVal is more suitable as a map of “daily assistance capabilities” rather than declaring a single unified score that covers everyone’s life.
For the product team, the most useful aspect of this evaluation is not creating another leaderboard, but rather adjusting the priority of features. Browser control, connectors, permission prompts, failure recovery, and result confirmation are often regarded as engineering details outside of the main model; however, in real-life tasks, it is precisely these elements that determine whether a particular assistance is effective or not.
AI Moving from chat tools to personal assistants, the real barrier is not about answering increasingly long questions, but about enabling people to avoid having to open multiple windows, miss out on a single piece of information, or make a mistake. LifeVal Breaking down "usefulness" into observable tasks, yet the ultimate standard still lies with the user: whether the task is completed properly, and whether they know where to stop in case of issues.
This will also change the way model updates are evaluated. Even if the new version scores a few more points on knowledge-based questions, if it is easier to overstep permissions, overlook restrictions, or fail to recover from web page errors during the process, the help provided to users may still be reduced. Only by taking into account the final delivery, necessary confirmations, and failure recovery can we get a closer approximation of the true quality of a personal assistant.
