Transformer Networks Optimized for ChatGPT Mobile
Siri and OK Google were initially a fun introduction to the promise of voice-based control, but we soon realized how carefully we must craft requests to get a useful response. The level of understanding we now see in ChatGPT would be much easier to use, but that capability has been limited to text interaction with cloud-based apps until recently. Now the compelling promise of ChatGPT and the ubiquity of cell phones is propelling a trend to make transformer networks for a ChatGPT mobile a reality, extending the power of large language models to everyone with a phone.
An obvious challenge is that the ChatGPT we know depends on trillions of parameters. Transformer networks of this size can only run in the cloud. Some suggest a hybrid model where a phone or other app does some of the work, connecting to the cloud for heavier duty inferencing. However, a casual phone-based user may not appreciate the long latencies and privacy risks inherent in a hybrid solution. A better approach would allow for running most or all of the transformer network load directly on the phone, turning to the cloud only for occasional anonymized search requests if needed.
To read the full article, click here
Related Semiconductor IP
- NFC wireless interface supporting ISO14443 A and B with EEPROM on SMIC 180nm
- DDR5 MRDIMM PHY and Controller
- RVA23, Multi-cluster, Hypervisor and Android
- CXL 3.0 Controller
- ECC7 Elliptic Curve Processor for Prime NIST Curves
Related Blogs
- Unveiling Ultra-Compact MACsec IP Core with optimized Flexible Crypto Block for 5X Size Reduction and Unmatched Efficiency from Comcores
- New Armv9 CPUs for Accelerating AI on Mobile and Beyond
- SiFive Accelerates RISC-V Vector Integration in XNNPACK for Optimized AI Inference
- Introducing Cortex-A320: Ultra-efficient Armv9 CPU Optimized for IoT
Latest Blogs
- The Evolution of AI and ML- Enhanced Advanced Driver Systems
- lowRISC Tackles Post-Quantum Cryptography Challenges through Research Collaborations
- How to Solve the Size, Weight, Power and Cooling Challenge in Radar & Radio Frequency Modulation Classification
- Programmable Hardware Delivers 10,000X Improvement in Verification Speed over Software for Forward Error Correction
- The Integrated Design Challenge: Developing Chip, Software, and System in Unison