1

Hello, I amCedricPerthuis,IamfromSonySantaMonicaandIamherewithTobiasBerghoff fromtheSCEadvancetechnologygrouptotalktoyouaboutMLAAonPS3.Tobiaswill firstpresentthePlaystation edgeMLAAlibraryandIwillthenshowyouhowweusedit andthebenefitswegotfromitinGodofWarIII. About2yearsago,AlexanderSiggraph paperaboutMLAAstartedcirculatingonour variousemailthreadswithintheSonygamestudios.Wewerestartingthefinalpush withabout6monthstogobeforeourgoldmaster.Westillhadmanygraphicsfeatures toaddandtheartistshadnt started polishing thegame.Therewere many unknowns andonly afewcertainties interm ofwhat we would make it into thefinalgame.MSAA was forusat that pointoneofacertainties.Still we somehow decided tomeasure the true cost ofMLAAinthegame.

Wewereabittakenbysurprise,thecostwasmorethanwhatwewereexpecting,total itwasabout6ms.ItwasnotbecauseofthehardwareMSAAitself,butratherbecauseof everythingelsewehadtodoaroundit. FewweekslaterweheardaboutourtechnologyteaminEuropeinvestigatingMLAA.We seizedtheopportunityandsentthemourlistofrequirements.

Thankyou,Cedric. TodayIllbetalkingaboutPlaystation EdgeMLAA,whichhasbeenavailableto Playstation 3developersforaboutayear.MostofthethingsIlltalkabouttodayarenew however,andhavenotyetbeenreleased.Thisworkwasdoneincollaborationwith Matteo Scapuzzi,whocantbeheretoday.

Duetotheextremelytighttimelimit,Illfocusontheimprovementswemadetoearly phasesofMLAA,andhowtheyaffectimagequality.Thisissortofagreatesthits selectionoftechniquesthatIthinkarehelpfulformostpostprocessAAsystems.There aremorenicefeaturesinthecode,butthisisallIcouldfitin. Ifyouareinterestedinmoredetails,pleasehavealookatthecoursenotes,orifyouare aPS3developer,goandgrabtheactualdocs.

Illstartwithsomegeneralbulletpoints aboutthesystem. [click] Naturally,werunonthePS3sSPUcores.BothourMLAApassesareactuallyhorizontal, withapseudotransposeinbetweentomakethatwork.Foreachpass,wehorizontally splittheframebuffer intoequalsizedchunksandletoneSPUlooseoneachofthem. Theentireoperationusuallytakesabout10msoftotalSPUtimefora720pbuffer, dependingonimagecomplexity.At30Hz,that5%ofyourtotalSPUbudget. [click] Inplaceprocessingwasmandatedbyourgameteams,tominimizememoryusage, [click] andthefastestintegrationintoagameIknowoftookabout2hours,soIsupposeour integrationpackagecantbethatbad. [click] Finally,Ishouldmentionthatthiscodehasshippedinawholelotoftiles,startingwith GodofWar3lastyear.ManytitlesofourWorldwideStudioshaveshippedwithitand manyexternalteamsareusingitaswell.

Afterthatshortmessagefromoursponsor,letstalkaboutimagequality. ThefirsttitleshippingwithMLAAwasGodofWar3,butoriginally,welookedintorealtime MLAAforKillzone 3.Bothofthesetitlesarebigtechnologyshowcasesforus,soapartfrom themhavingtobereallygoodgames,wealsoneedthemtolookgreat.Thus,anyAAsystemwe useneedstobeasignificantimprovementoverwhatwasusedbefore,whichis2xMSAAinboth cases. [click] Togetthisquality,wenotonlywanttosmoothfeaturesthatMSAAmisses,suchasalphatested geometry,butthegoalistofiltereverybitofopaqueandtransparentgeometry,allreflections, refractions,shadows,etc.Andwewanttodothisaftertonemapping.ThisalltranslatestoMLAA beingappliedquitelateintheframe,andtotheinputsbeingquitevariedintermsofluminance, whichrequireshightemporalstability.AndtemporalstabilityisnotMLAAsgreateststrength. [click] Furthermore,jaggies andartefactsinanotherwisecleanimageareverynoticeable,leadingto whatIcalltheAAuncannyvalley.Thismeansthattheperceivedlossofqualityduetoan artifact getsworseastheimagegetsbetter,makingthisverymuchanuphillbattle. [click] Asastartingpoint,weusedthecolorversionofMLAAthatAlexproposedinhisoriginalpaper, whichdoesntseemtobewhatmostpeopleareusing.Itgaveusbetterquality,sowewentwith thismoreexpensiveapproach.Thisisinterestingbyitself,butIdonthavethetimetogointo thishere. Sowithoutfurtherrambling,letshavealookattheimprovements.

ThisistheGodofWar3spikeroom,whichhasbeenourgotoimagesincetheinitial developmentofthesystem.Thisisnotactuallywhatthisroomlookslikeinthefinal game,butIllbeusingthatasthesourceimage.

Wellstartbylookingatthiscutout.

Letsstartwiththeedgedetection,whichisthefirststepofthealgorithm,andallother operationsdependonit.Sinceitsusedasanon/offswitchfortherestofthe operations,changesintheedgevaluesaregreatsourcesofnoisethatyoullneverget ridofagain.Inmyexperience,theworstantialiasingartefactsarethosethathaveno obviouscauses,astheywillattracttheviewersattentionwithoutallowingthemto easilydiscardthemasartefacts.Thisisexactlywhatanoisyedgedetectionwillgiveyou. [click] Asareferencepoint,Idliketostartwiththedefaultedgedetectionalgorithmoften used,whichisperchannelthresholding.Theideaistocomputetheabsolutedifferences betweenthecolor channelsoftwopixelsandifanyofthemarelargerthansomeglobal threshold,thatsanedge.Itsverysimple,itsveryfast,butnotreallyproductionquality. Wecallitabsolutethresholdingforsoontobeobviousreasons.

10

Thisistheresultwithabsolutethresholding andathresholdof31.Asyoucantell,the jaggies aremostlygone,butsoisalotofthedetail. Ontherightsideyoureseeingtheedgebuffer,wherehorizontaledgesaregreen, verticaledgesarered.Noticethatthedarkbackgroundregionsaremostlyedgefreeand thebrightregionsareprettymuchalledges,exceptforthecenter oftheflame,where thecolor isclamped.Onlyinthemiddleofourluma rangedowegetreliabledetection.

11

HerewehavetheresultwegetfromwhatIcallrelativethresholding.ImsureImnot thefirstonetouseit,butmygooglefudidntgetmeanywhere.ItsprettysimpleandIll explainitinasecond.Theimportantparttonoticeisthatwenowhavegooddetection overtheentireluma range,whichalsotranslatesintoanoticeablysharperimageonthe left.Theedgedetectionresponseisalmostinvertedcomparedtothepreviousslide. Ihavetopointoutthattheedgeofthebladeisabitlesssmooth,becausethereis actuallyadoubleedgeinthesourceimageherewhichabsolutethresholding doesnt pickup,butwhenitispickedupcausesblendingtobelesssmooth.

12

So,howdoesrelativethresholding work?Itsatwopartprocess,thefirstofwhichprepareseachpixelandthesecondperformstheactual comparison.Theoverallideaistocomputethethresholdvalueperpixelpair,insteadofhavingoneglobalvaluefortheentire image. [click] Westartwithtwoinputpixels,butinrealityyouddothisforallpixelsbeforemovingontothesecondphase. [click] Foreachpixel,wefirstpickthenumericallylargestchannel. [click] Followingthat,wescalethosechannelsbyauserdefinedvalue,whichdeterminesthesensitivityoftheedgedetection.Wethen applyalowerbound totheresult,whichisusedtodealwithverydarkregions.Thegreenchannelheregetsadjustedbythis.Ifwedidntdothis, wedgetlotsofedgesas colors andthusthresholdsgotowardszero. [click] Theresultsarethenstoredbackintothealphachannelsofthesourcepixels. [click] Thetestitselfstartswithtwopreparepixels. [click] Wecomputetheabsolutedifferenceofthecolor channels,justlikeintheabsolutethresholding case. [click] Againpickthenumericallylargestchannelofthat, [click] butnowwealsopickthesmaller ofthetwothresholds. [click] Finally,wecompare thetwoandifthethresholdissmaller,wehaveanedge. Weusethecolor channelsandnotluma tobeabletodetectchroma edges.Thesinglethreshold,asopposedtohavingperchannelthresholds,comes mostlyfromstoragerequirements,asweonlyhave8bits,butwealsodidntwanttogetanedgeiftherelativedifferenceisonlylargein unimportantchannels.Thereasonwepickthesmallerthresholdistomakethetestsymmetrical. Averagingthethresholdsor takingthemaxworks aswell. AsIsaid:Prettysimple.Andactuallyprettyfast.

13

Withthescaleandlowerbound whichwecallthebase,wehavetwoparametersto tweak,whichisquiteimportantforus.Notwogamesarethesame,andforsometitles, evendifferentlevelsbenefitgreatlyfromusingdifferentsettings. ThiswastheonlyrealchangewemadeforGodofWar3andtheEdgeMLAAcodewe released,soletsmoveontonewthings.

14

Heresaslightlylargercutout fromthespikeroom.PleasekeepaneyeonKratoss face.

15

Thisisabsolutethesholding again.Youllnoticeitsquiteblurry.

16

RelativeThresholding.Better,butwithitsownartifacts.

17

Andournextimprovement.ThebladeinfrontofKratoss chestprobablybenefitsthe most,butthereddetailtexturesgainsomesharpnessandwecanagainseeKratoss fashinable eyeliner.

18

Again,thesourceforcomparison. So. Howdidwedothat?

19

Thewayitisusuallyimplemented,MLAAhasnoconceptofadjacentedges. [click] Ifoneedgebeginsrightwhenanotheroneends,thosetwoareconsideredtobeone edge. [click] Astartandanendpointarethendeterminedforthisfusededgeandblendingis performedovertheentirestructure.Clearly,thisisnotideal. [click] Oursolutionistosplitanedgeifthecolor gradientsignchanges,whichweapproximate bylookingatthesumofcolor channels. [click] Ivetriedahandfulofotherapproachestodeterminethissignchange,butnoneofthem reallyimprovedtheresultoverthisverysimpleandfastapproach.Itakethattomean theywereallequallybad,soImsuresomesmartpersonwillfindamuchbetterwayto dothis.

20

Toillustratetheeffect,Idliketoshowyouthecheckerboardpatternagain. [click] PerformingjustanormalhorizontalMLAAdoesntreallygetusanyantialiasing. [click] Onlywhenwesplitthecompositeedge,dowegetthedesiredresult. Ivealreadyshownyoutheimprovementthismadeforthespikeroom,butmaybeyou foundthatabitunderwhelming,soletmeshowyouwhywedidthis.

21

Thisisaseriouslyundersampled metalgrillfromKillzone 3. [click] Withthepreviousmethod,wedlosemostoftheorientationandsimplyget,well,noisy stuff. [click] Oncewesplitthefeatures,wegetamuchnicerresult.NoticethatMLAAactuallyworks prettywellagainstspecular aliasinghere,whichisrelated.Thereasonisthatforreally shortfeatures,MLAAapproximatesablur.Aswesplitfeaturesaggressively,wegetthese shortfeatureshere,whichisamuchmorestablewaytodealwithhardtofilterinputs.

22

OK,onemore.SometimeafterGodofWar,Killzone 3wasgettingreadytoship.They hadbeenusingourMLAAcodeforawhile,butwhenIshowedthemthefeature splittingIvejustshownyou,alotoflastminuteactivityfollowed. Thereasontheywereinterestedingettingsomemoreworkdoneisthatthegamehas veryhighfrequency,highcontrasttextures.Justlookatthered/blackdetailsinthose pits.GodofWarhasaverydifferentartstyle,thiswasmuchlessofanissue. Thisframe,however,isnothandledsufficientlywellbyrelativethresholding.

23

...asyoucanseehere.

24

Sothecurrentresultwasntreallywhatwewantedandwehadtorevisitedge detection. [click] TheKillzone guyshadbeenexperimentingwithfeedingadditionalnoncolor data intotheedgedetectionbefore,butwedecidedtogiveitanothershotliterally10 daysbeforegoinggold. Astheyareusingadeferredrenderer,thereareallkindsofbufferslyingaround thatcouldbeusedforedgedetection. [click] Theyendedupfeedingpartofthelightingcomputationintothealphachannel, whereMLAAwouldpickitup.Thatwaynoadditionalstorageorbandwidthwas required. [click] Therealquestion,however,ishowtocombinethetwoedgesources.

25

Herestheedgebufferforjustthelightingdata,usingrelativethresholding tofindthe edges.

26

Thisistheactualinput,somewhatcontrastenhancedtoimprovevisibilityonthe projectorhere.

27

UpthereisacutoutIwanttoshow.

28

Thefirstideascentered aroundeitherdroppingcolor edgesaltogether,orusingthe lightingasanothercolorchannel. [click] Theproblemwiththeseapproachesisdemonstratedontheleft.Ifanedgethatdoes notexistinthecolor dataintersectsanedgethatdoesexistinthecolor data,the algorithmwilltrytocreateasmoothtransitionbetweenthetwo,effectivelydamaging thevisibleedge. [click] Soweneedtobasethefinaldecisionifsomethingisanedgeontheactualcolor data. Whichiswhatwedid.

29

Oursolutionwastoperformtwoedgedetectionstages. [click] First,weprocessthenoncolor data.Thisgivesusanedgemaskwhichwefeedintothe secondstage. [click] Herewelookatthecolor data,butifwepreviouslyfoundanoncolor edge,weincrease thesensitivity.Weknowsomethingstheresowelookextraclosely. [click] Wecouldhavegoneforonlylookingforcolor edgeswheretherearenoncolor edges, butthatwouldhavemissedtoomanyimportantfeatures.Still,wecannowtonedown thegeneralsensitivityandstillgetprettymuchalledgeswewant. [click] Thereallycoolthingaboutthis,however,isthatthisfirstpasscanbeprettymuch anythingyouwant.YoucouldrunaSobel,whichdoesntevengivethebinaryedgeswe needlater.Orworkonnormals,orobjectIds,whichweusedinSOCOM:SpecialForces.

30

HeresanexamplefromSOCOM,againcontrastenhanced, [click] thenthematerialIds, [click] thecolor edges [click] andthecombinededges.Notethattherearestillholesinthecombinededges,sincethe color valuesareextremelycloseorequalinthesourceimage.Maybewecouldpatch thesethese withthenoncolor data.

31

Heresthenonpredicatedresult.Noticetheunfilteredblacklines.

32

Andthepredicatedresult.

33

ThisisagainKillzone 3withoutpredication.

34

...andwithpredication.Ididntperfectlytweaktheparametersfortheimagehere,butI thinktheimprovementisprettyclear.

35

Alright,letswrapthisup. [click] ThemessageIwantedtobringacrosstodayisthathowyoufindyouredgesandwhatyoudowiththemis extremelyimportantforthefinalresult.AAsystemsareinthebusinessofreducinghighfrequencynoise, so youwanttoaddaslittletotheproblemaspossible. [click] Forusthatmeansthatweusuallytrytofindthebestqualitysolutionwithinreason,andthenoptimize thattillwemeettheperformancegoals.Weneverchangedparameterstocompromisequalityforspeed. [click] SowhatIpresentedtodayisbasicallyasmallselectionofhackstomakeMLAAproducebetterresults. NoneofthesearewithouttheirownissuesandIpointedoutafewofthem,so inmyopinion,postfx AA remainsaveryinterestingfieldofresearch. [click] Onereasonforthatisthatitsnotreallyawellunderstoodproblem,atthispoint.Otherpeoplemay disagreewiththat,butIcertainlydontknowhowtomaketheperfectpostfx AAsystem. [click] ThisalsomeansthatwetendtoclassifythingsastypicalMLAAartifactsthatareactuallysolvable problemsorjustbugs. [click] Still,attheendoftheday,wecananddoshipbigAAAtitleswithitsuccessfully,becauseitsareallysmart idea,andImlookingforwardtoseeingwhereitgoes.

36

Andthatsallfromme.Idliketothentheseteamsandindividualsandyouforlistening. CedricwillnowbetalkingabouttheGodofWar3integrationanddemoittoyou,soyou cangetsomeintuitionabouthowthislooksinarealgame.

37

38

39

40

41

42

43

44

UsesrelativelyadvancedPS3features RarelyusedmonitoringfeatureoftheSPUkernel,andGPUbackendreport command AllowsperfectcouplingbetweenGPUandSPUMLAAtask

45

Shouldrequiretokeepalldrawdatafor1moreframe Okforposteffects,verylittledata Toodifficultforgeometriesduetothedeeprendererpipelineandthememory requirements GodofWarIIItrick: ForpostMLAAgeometries,usenextframegeometry

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61