How to perform meaningful benchmarks on FPGAs from different vendors
By Seyi Verma, Altera Corporation
Programmable Logic DesignLine (05/28/08, 01:39:00 PM EDT)
As a design engineer using FPGAs, how do you choose which FPGA is the best for your application? Every engineer faces this problem and is inundated with endless, self-promoting marketing claims that many times are taken with a "pinch of salt". Every FPGA vendor asserts that their part is this much faster, takes this much less time to compile, and consumes this much less power as well as claiming numerous other technical specifications. After all, any company can create artificially-tailored benchmarks that can put their FPGA in a better light when compared to that of the competition. So the question is: which benchmarking claims are accurate and which are marketing claims?
Altera evaluates and benchmarks its FPGAs in conjunction with its Quartus II software at every process node on a suite of comprehensive and representative customer designs. These customer designs are collected from a variety of market segments, such as networking, telecommunications, wireless, and consumer applications, and a variety of implementation technologies such as ASICs, gate arrays, and FPGAs from other vendors. These customer designs (which are maintained in a secure database) are used to understand how real customer designs perform against the current architecture and with that of the competition. Designs that do not optimize well on Altera's FPGA architecture are evaluated on over 150,000 experiments; enhancements to future incarnations of the FPGA architecture or design tools are then recommended based on the evaluations performed.
Based on the evaluation and benchmarking processes introduced above – and the fact that there is no standardized FPGA benchmarking processes – Altera has created what we believe to be a "fair and unbiased" benchmarking methodology, which is based on a set of real customer designs and endorsed by industry experts. But there's a problem: since the existing designs are proprietary to our customers, we cannot provide these designs for evaluation of Altera FPGAs by new/other customers. Although everyone understands the reasoning behind this, it still serves to amplify end-user skepticism. If I was a potential customer of Altera and was told that Stratix III FPGAs are on average 35% faster than Virtex-5 FPGAs, compile three times faster than Virtex-5 FPGAs, and provide 95% core utilization on average, then without the means of evaluating this myself, as an engineer, I would be very skeptical!
Programmable Logic DesignLine (05/28/08, 01:39:00 PM EDT)
As a design engineer using FPGAs, how do you choose which FPGA is the best for your application? Every engineer faces this problem and is inundated with endless, self-promoting marketing claims that many times are taken with a "pinch of salt". Every FPGA vendor asserts that their part is this much faster, takes this much less time to compile, and consumes this much less power as well as claiming numerous other technical specifications. After all, any company can create artificially-tailored benchmarks that can put their FPGA in a better light when compared to that of the competition. So the question is: which benchmarking claims are accurate and which are marketing claims?
Altera evaluates and benchmarks its FPGAs in conjunction with its Quartus II software at every process node on a suite of comprehensive and representative customer designs. These customer designs are collected from a variety of market segments, such as networking, telecommunications, wireless, and consumer applications, and a variety of implementation technologies such as ASICs, gate arrays, and FPGAs from other vendors. These customer designs (which are maintained in a secure database) are used to understand how real customer designs perform against the current architecture and with that of the competition. Designs that do not optimize well on Altera's FPGA architecture are evaluated on over 150,000 experiments; enhancements to future incarnations of the FPGA architecture or design tools are then recommended based on the evaluations performed.
Based on the evaluation and benchmarking processes introduced above – and the fact that there is no standardized FPGA benchmarking processes – Altera has created what we believe to be a "fair and unbiased" benchmarking methodology, which is based on a set of real customer designs and endorsed by industry experts. But there's a problem: since the existing designs are proprietary to our customers, we cannot provide these designs for evaluation of Altera FPGAs by new/other customers. Although everyone understands the reasoning behind this, it still serves to amplify end-user skepticism. If I was a potential customer of Altera and was told that Stratix III FPGAs are on average 35% faster than Virtex-5 FPGAs, compile three times faster than Virtex-5 FPGAs, and provide 95% core utilization on average, then without the means of evaluating this myself, as an engineer, I would be very skeptical!
To read the full article, click here
Related Semiconductor IP
- MACsec Engine optimized for 10Base-T1S/T1L applications
- Dual Port, Voltage Scaling Register File Compiler on GF22FDX+
- Single Port, Voltage Scaling, High Speed SRAM Memory Compiler on GF22FDX+
- I3C Full Controller / Target
- I3C HCI Host Controller
Related Articles
- Xilinx responds to Altera's FPGA benchmarks
- SoCs: DSP World, Cores -> Benchmarks elusive in DSP world
- Using the ARM Cortex-R4 for DSP, part 1: Benchmarks
- Designing FPGA Based Reliable Systems Using Virtex-5 System Monitor
Latest Articles
- PULSAR-V: I/O-Aware Post-Update Attestation for RISC-V
- PVTSizing 2.0: A Fast and Reliable Analog Circuit Sizing Operator for Agentic EDA Workflows
- An Agentic Python Framework for LLM-Driven UVM Testbench Synthesis with Closed-Loop Compiler Feedback
- Reconfiguring System-Level Protection Through Bounded Delegation in RISC-V MCUs
- SkipLane: Write-Once, Conflict-Free Accumulation for Zero-Skipping CNN Engines on FPGAs