FPGA Based Processor

for Hubble Space Telescope

Autonomous Docking – A Case Study

Jonathan F. Feifarek
jonathan.feifarek@lmco.com

Timothy C. Gallagher
timothy.c.gallagher@lmco.com

Lockheed Martin Space Systems Co.

Courtesy NASA GSFC

Feifarek 1 MAPLD 2005/A220

 Background: Need for Hubble Repair
●
4 / 1990: Hubble Space Telescope (HST) launch
●
12/1993: SM* 1- Corrective COSTAR, WFP Camera2
●
2 / 1997: SM 2 – Add NICMOS, STIS, Thermal Blankets
●
10/1997: Hubble Operations Extended from 2005 to 2010
●
12/1997: SM3A Replace 6 Gyros, 3 Fine Guidance Sensors
●
3 / 2002: SM3B Replace Solar Panels, NICMOS Coolant
●
3 / 2003: SM 4 Cancelled Following Columbia Disaster
●
6 / 2004: Hubble HRV Request For Proposal Issued
●
8 / 2004: Lockheed Martin awarded HST Robotic Vehicle (HRV)
●
12/2007: Target HRV Launch Date
* SM = Service Mission
Feifarek 2 MAPLD 2005/A220
 HRV Mission : Autonomous Docking

Mission Phase System Approach

Requirement

• Orbit phasing with HST

• HRV checkout
Pursuit • Range from HST for initial sensor acquisition

• HST approach with safe-hold points

• Acquire sensor data on HST orientation and rotation rate
Proximity Ops

• Rate matching with HST

• Maneuver to HST capture point
Approach

• Capture HST:
– Robotic Arm Captures HST Grapple Fixture
Capture/Berth – Berth to HST aft interface

Feifarek 3 MAPLD 2005/A220

 Vision Processing Algorithm Selection Criteria
●
Implementation Concerns - Computational Intensive
– Field Programmable Gate Array (FPGA)
– Flight Computer, DSP Processor
– Combination
●
Implementation Approach
– All: Conventional Programming Languages
– FPGA High-Order Languages (HOLs)
– FPGA Register Transfer Logic (RTL) in VHDL or Verilog
●
Error-prone
●
Time consuming (calendar time plus engineering cost)
●
Difficult to achieve bit accurate & cycle accurate
operations using hand-coded conversions

Feifarek 4 MAPLD 2005/A220

 Vision Processing Algorithm Selection Results
●
FPGA Reconfigurable Architecture Chosen
– Searched Internet and Conference Proceedings for comparisons
between Processors and FPGA Reconfigurable Computer (RCC)
●
Space Based RCC technology leaders such as Los Alamos
National Labs1 and NASA2 noted FPGA system performed
between 10-1000x faster then processors
●
Many other references on FPGA based accelerated image
processing from University studies3,4
●
Microprocessor Embedded in FPGA
– Allows rapid evaluation of architecture performance
– Can host large amounts of existing code such as decision logic
and complex sequential math
– For certain algorithms Floating Point is more efficiently
implemented in processor code then in gates

Feifarek 5 MAPLD 2005/A220

 Vision Processing Algorithm Selection Results
●
FPGA Implementation: Combination of HOL, RTL
– HOL (Celoxica Handel-C) for fast and efficient implementation
– Provided fast development cycle needed for mission
●
Quickly ported math libraries & existing C++ code
●
Performance matched RTL speed, area ; slower than hand-code
●
Highest speed increase from hand floorplanning
– RTL for IO Wrapper, IO reuse, and custom-optimized code
– Combined the benefits of all worlds
– Microprocessor Implementation
●
Incorporated Xilinx MicroBlaze ™ Core in FPGA
●
Xilinx tools: Platform Studio© SDK / EDK suite
●
Used gnu© C compiler / “gdb” debugger

Feifarek 6 MAPLD 2005/A220

 Vision Processing FPGA Development Flow

C Algorithm
FPGA Acceleration
C to RTL
Implementation Generate human-readable
Provide rapid iteration VHDL and Verilog for 3rd
of partitioning decisions party synthesis
throughout flow
C to FPGA
Verification Direct implementation to
Drive continuous device optimized
system verification programmable logic
from concept to
hardware

Used with permission of Celoxica, Inc.

Feifarek 7 MAPLD 2005/A220

 Vision Processing Card (VPC) Block Diagram
RAM Serial Camera
Raw images Pixels Edge Enhanced Images
Program,
Data Memory t+1 t Loading Loading t

FPGA
Memory Manager Memory Manager
Xilinx microBlaze™ Front End
MicroProcessor Core Enhanced
Image Patches Pixels
Image
Pixels
Processor
Project Model Points
(3D to 2D Images) Image Points Pyramidal
Lukas Kanade Downsampling /
Trackers Edge
Enhancement
Compute New Image Points
Pose

Edge Single FPU Instance With Multiple Software Invocations

Project Edges Edge Finder Custom Floating Point Unit (FPU)

Compute New
Operation Request Operator Scaler,
Best Fit Edge uBlaze Matrix
Pose (Iterative) In/Out Operands In , */ -
Results Out Data
Convert
uBlaze Software Libs
Output Pose
Floating Point Unit
uBlaze Hardware FPU Data Pipeline
Control
Feifarek 8 MAPLD 2005/A220
 Vision Processor Card Architecture

Port A Port B Port C Port D

SerDes SerDes SerDes SerDes

SDRAM

SDRAM

SDRAM

SDRAM

SDRAM

SDRAM

SDRAM

SDRAM
SRAM

SRAM

SRAM

SRAM
COP A COP B COP C COP D
Xilinx V2 Xilinx V2 Xilinx V2 Xilinx V2

Internal PCI

Common Interconnect Bus

SRAM
PCI-PCI
Bridge /
Flash
Config

Power
Switch

PCI Connectors J8

Feifarek 9 MAPLD 2005/A220

 VPC Engineering Development Board

Used with permission of SEAKR Engineering, Inc.

Feifarek 10 MAPLD 2005/A220

 VPC SEU Approach
●
Main SEU Mitigation: Dual Voting at FPGA output
– Detects SEE's but cannot correct for them
– Tight power restrictions (thermal reasons) restrict triple voting
– Vision Processing Algorithm tolerant of drop-outs
●
Multiple camera views / algorithms into Kahlman filter
●
HRV mission uses very low rate docking (1 inch / sec)
●
SEU Correction at FPGA-to-Memory Interfaces
●
Microprocessor returned to Reset State after each image
●
Algorithm memory only 1 image deep; flushes SEU effects
●
Voting, Configuration Scrubbing Performed in Rad Hard Part
●
Analysis Shows Low SEE Rate (1 effective upset / 10 hours)

Feifarek 11 MAPLD 2005/A220

 VPC Sizing Results for NFIR Algorithm
Single LK Tracker

LUTs Flip Flops Multipliers BlockRAMs

MicroBlaze Processor 2700 2000 4 33
Front End + 8000 4000 0 42
LK Tracker 7000 2700 19 24
Total 17700 8700 23 99
Available 67584 67584 144 144
Percentage Utilized 26% 13% 16% 69%

Quad LK Trackers

LUTs Flip Flops Multipliers BlockRAMs

MicroBlaze Processor 2700 2000 4 33
Front End + 8000 4000 0 42
LK Tracker (4) 13580 10580 70 68
Total 24280 16580 74 143
Available 67584 67584 144 144
Percentage Utilized 36% 25% 51% 99%

Feifarek 12 MAPLD 2005/A220

 VPC Performance Results for NFIR Algorithm

Function Timed Cycles/Loop Loops Total Cycles

Project Model Points 26000 1 26000
Lktracker (hardware) 2000000 1 2000000
FindExtrinsic 3078000 1 3078000
Project edges 120000 3 360000
FindEdges (hardware) 400000 1 400000
Project ellipses 80000 3 240000
computeAllFis 180000 2 360000
computeVsumCsum 280000 2 560000
computeAlpha 230000 2 460000
UpdatePose 6000 3 18000
getAllErrors 240000 3 720000
Total 6640000 8222000

Feifarek 13 MAPLD 2005/A220

 VPC Performance Results for NFIR Algorithm (cont.)

FindExtrinsic timing Cycles/Loop Loops Total Cycles

Normalize 73000 1 73000
SVD6x6 170000 3 510000
SVD3x3 35000 1 35000
FindHomography 1100000 1 1100000
ProjectPoints 190000 4 760000
Rest 600000 1 600000
Total 2168000 3078000

Feifarek 14 MAPLD 2005/A220

 Summary: Lessons Learned
●
Using OpenGL algorithm for development hampered design
●
Parallel PC board and FPGA designs helped meet schedule
●
Using FPGA’s was key to meeting speed requirements
●
Use of microprocessor core reduced development time
●
Early allocation of algorithm to hardware/software paid off
●
Use of HOLs made implementing complex tasks possible
●
Engage expert tool user on team (MicroBlaze, Handel-C)
●
Having reference software / test data eased verification
●
Benefited from small, enthusiastic, tight knit team
●
Worked around MicroBlaze libraries bugs with custom logic
Feifarek 15 MAPLD 2005/A220
 References
●
(1) “A Space Based Reconfigurable Radio”, Michael Caffrey,
Los Alamos National Laboratory, MAPLD September 2002
●
(2) “Developing Reconfigurable Computing Systems for Space Flight
Applications”, Thomas P. Flatley,
NASA Goddard Space Flight Center Greenbelt, Maryland 20771
●
(3) "Implementing Image Applications on FPGAs," B. Draper, R. Beveridge, W.
Böhm, C. Ross and M. Chawathe. International Conference on Pattern
Recognition, Quebec City, Aug. 11-15, 2002.
●
(4) “Performance of Reconfigurable Architectures for Image-Processing
Applications”, Domingo Benitez, University of Las Palmas G.C.,
Journal of Systems Architecture: the EUROMICRO Journal, September 2003

Feifarek 16 MAPLD 2005/A220