AI - Sunday, October 20, 2024: Notable and Interesting News, Articles, and Papers
AI - Sunday, October 20, 2024: Notable and Interesting News, Articles, and Papers
Commentary and a selection of the most important recent news, articles, and papers about AI.
Today’s Brief Commentary
Today, I include some articles discussing how Google, Microsoft, and AWS are making deals to use small nuclear reactors to power their presumably AI data centers. Is this the right way to go when we have sustainable methods like solar and wind? Much of the sustainability discussion in the industry talks about new chips and devices that use much less energy to do more compute. While good, are they just burning through this energy elsewhere? This isn't doing "more with less." This is doing "more with the same and even more eventually with nuclear power."
General News, Articles, and Analyses
Wimbledon Scrapping Linespersons for AI in 2025
https://uni-watch.com/2024/10/14/wimbledon-scrapping-linespersons-for-ai-in-2025/
Author: Phil Hecken
(Monday, October 14, 2024) “The venerable Wimbledon tennis tournament, one of four “Grand Slams” in tennis, will be replacing line judges at next year’s championships. They will be replaced by line-calling powered by artificial intelligence, the All England Club confirmed last week.”
Treasury Announces Enhanced Fraud Detection Processes, Including Machine Learning AI, Prevented and Recovered Over $4 Billion in Fiscal Year 2024
https://home.treasury.gov/news/press-releases/jy2650
(Thursday, October 17, 2024) “Today, the U.S. Department of the Treasury announced that its latest efforts in taking a technology and data-driven approach to fraud and improper payment prevention enabled the prevention and recovery of over $4 billion in fraud and improper payments this fiscal year (FY) (October 2023 – September 2024), up from $652.7 million in FY23. This increase reflects dedicated efforts by Treasury’s Office of Payment Integrity (OPI), within the Bureau of the Fiscal Service (Fiscal Service) to enhance its fraud prevention capabilities and expand offerings to new and existing customers.”
Nvidia just dropped a new AI model that crushes OpenAI’s GPT-4—no big launch, just big results | VentureBeat
Author: Michael Nuñez
(Thursday, October 17, 2024) “Nvidia quietly launched a groundbreaking AI model that surpasses OpenAI’s GPT-4 and Anthropic’s Claude 3.5, signaling a major shift in the competitive landscape of artificial intelligence.”
Case Study: Should We Deploy a Gen AI Salesbot? | Harvard Business Review
https://hbr.org/2024/11/case-study-should-we-deploy-a-gen-ai-salesbot
Authors: Jill Avery and Thomas Steenburgh
(Friday, November 1, 2024) “After seeing gen AI’s impressive capabilities demonstrated at a conference, CEO Jeannie Weiss is excited to put them work at her digital marketing firm, PulsePoint. A cutting-edge salesbot would allow her to shrink her staff while delivering more-effective and -personalized service to her clients. Two of her top executives have qualms, however—and so does her biggest customer. But if she puts the brakes on the project, will rivals who move faster on gen AI open a huge lead over PulsePoint?”
Energy and Sustainability
Hungry for Energy, Amazon, Google and Microsoft Turn to Nuclear Power
Authors: Ivan Penn and Karen Weise
(Wednesday, October 16, 2024) “Large technology companies are investing billions of dollars in nuclear energy as an emissions-free source of electricity for artificial intelligence and other businesses.”
Google Inks Deal for Nuclear Power for AI Tech, Data Centers
https://aibusiness.com/verticals/google-inks-deal-for-nuclear-power-for-ai-tech-data-centers
Author: Liz Hughes
(Friday, October 25, 2024) “Google has signed a deal for what the company called the world’s first corporate agreement to purchase nuclear energy from multiple small modular reactors (SMRs) to power AI technologies. Set to be developed by Kairos Power, Google said the first phase would bring the first SMR online “quickly and safely” by 2030, with additional reactors deployed through 2035.”
Analysis and Insights from The Futurum Group
GlobalFoundries, Silicon Catalyst to Boost Semiconductor Startups - The Futurum Group
https://futurumgroup.com/insights/globalfoundries-silicon-catalyst-to-boost-semiconductor-startups/
Author: Dr. Bob Sutor
(Wednesday, October 16, 2024) “GlobalFoundries partners with Silicon Catalyst to accelerate semiconductor startups’ growth, focusing on AI, IoT, and quantum computing.”
Technical Papers, Articles, and Preprints
[2410.12881] MIND: Math Informed syNthetic Dialogues for Pretraining LLMs
https://arxiv.org/abs/2410.12881
Authors: Akter, Syeda Nahida; Prabhumoye, Shrimai; Kamalu, John; Satheesh, Sanjeev; Nyberg, Eric; Patwary, Mostofa; Shoeybi, Mohammad; and Catanzaro, Bryan
(Tuesday, October 15, 2024) “The utility of synthetic data to enhance pretraining data quality and hence to improve downstream task accuracy has been widely explored in recent large language models (LLMs). Yet, these approaches fall inadequate in complex, multi-hop and mathematical reasoning tasks as the synthetic data typically fails to add complementary knowledge to the existing raw corpus. In this work, we propose a novel large-scale and diverse Math Informed syNthetic Dialogue (MIND) generation method that improves the mathematical reasoning ability of LLMs. Specifically, using MIND, we generate synthetic conversations based on OpenWebMath (OWM), resulting in a new math corpus, MIND-OWM. Our experiments with different conversational settings reveal that incorporating knowledge gaps between dialog participants is essential for generating high-quality math data. We further identify an effective way to format and integrate synthetic and raw data during pretraining to maximize the gain in mathematical reasoning, emphasizing the need to restructure raw data rather than use it as-is. Compared to pretraining just on raw data, a model pretrained on MIND-OWM shows significant boost in mathematical reasoning (GSM8K: +13.42%, MATH: +2.30%), including superior performance in specialized knowledge (MMLU: +4.55%, MMLU-STEM: +4.28%) and general purpose reasoning tasks (GENERAL REASONING: +2.51%).”
[2410.13720] Movie Gen: A Cast of Media Foundation Models
https://arxiv.org/abs/2410.13720
Authors: Polyak, Adam; Zohar, Amit; Brown, Andrew; Tjandra, Andros; Sinha, Animesh; Lee, Ann; Vyas, Apoorv; Shi, Bowen; Ma, Chih-Yao; ; ...; and Du, Yuming
(Thursday, October 17, 2024) “We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user's image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models. All videos from this paper are available at https://go.fb.me/MovieGenResearchVideos.”
Related Articles and Papers
Meta Movie Gen
https://ai.meta.com/research/movie-gen/
“Movie Gen sets a new standard for immersive AI content. Our latest research breakthroughs demonstrate how you can use simple text inputs to produce custom videos and sounds, edit existing videos or transform your personal image into a unique video.”