Video processing

Reasonsforprocessing

TheaspectratiooftraditionalTV(green)andcommonmoviescreens.Aspectratioisusedtodescribetheratioofvideoscreenstoscreenelements.TheaspectratioofthetraditionalTVscreenis4:3(1.33:1).TheaspectratioofHDTVis16:9(1.78:1).Theaspectratioofthe35mmfilmnegativeisabout1.37:1.

Althoughthepixelsonthecomputerscreenaremostlysquare,thepixelsofdigitalvideoareusuallynot.Forexample,CCIR601,thedigitalstorageformatusedforPALandNTSCsignals,anditscorrespondingnon-equalsquarewidescreenformat.Therefore,NTSCstandardDVimagesrecordedwith720x480pixelsmaybecomea4:3aspectratioscreenduetotherelatively"thin"pixelformat,orviceversabecausethepixelformatis"fat"andbecome16:9.Picture.

Commonformats

AVI---AVIfile

AVIistheabbreviationofAudioVideoInterleaved.ItisadigitalaudioandvideofileformatthatconformstotheRIFFfilespecificationdevelopedbyMicrosoft.ItwasoriginallyusedintheMicrosoftVideoforWindows(VFW)environmentandhasbeendirectlysupportedbymostoperatingsystemssuchasWindows95/98andOS/2.TheAVIformatallowsvideoandaudiotobeinterleavedandplayedsynchronously.Itsupports256colorsandRLEcompression,buttheAVIfiledoesnotlimitthecompressionstandard.Therefore,theAVIfileformatisonlyusedasastandardonthecontrolinterfaceandisnotcompatible.Itusesdifferentcompressionalgorithms.ThegeneratedAVIfilemustbeplayedoutwiththecorrespondingdecompressionalgorithm.CommonlyusedAVIplaybackdriversaremainlyVideo1inMicrosoftVideoforWindowsorWindows95/98,andIndeoVideofromIntel.AVIfilesaremainlyusedonmultimediadiscstosavevariousimageinformationsuchasmoviesandTVs.SometimestheyalsoappearontheInternetforuserstodownloadandappreciatethewonderfulfragmentsofnewmovies.

MPEG/MPG/DAT---MPEGfile

MPEGfileformatisaninternationalstandardformovingimagecompressionalgorithms,whichuseslossycompressionmethodstoreduceTheredundantinformationinthemovingimage,whileensuringadynamicrefreshrateof30framespersecond,hasbeensupportedbyalmostallcomputerplatforms.TheMPEGstandardincludesthreeparts:MPEGvideo,MPEGaudio,andMPEGsystem(video,audiosynchronization).TheMP3audiofileintroducedaboveisatypicalapplicationofMPEGaudio,andVideoCD(VCD),SuperVCD(SVCD),DVD(DigitalVersatileDisk)isanewtypeofconsumerelectronicproductsproducedbyfullyadoptingMPEGtechnology.TheMPEGcompressionstandardisdesignedformovingimages.Itsbasicmethodistocollectandsavethefirstframeofinformationinaunitoftime,andthenonlystorethepartsoftheremainingframesthathavechangedrelativetothefirstframe,soastoachievethepurposeofcompression.Twobasiccompressiontechnologiesareused:motioncompensationtechnology(predictivecodingandinterpolationcode)toachievetimecompression,andtransformdomain(discretecosinetransformDCT)compressiontechnologytoachievespatialcompression.TheaveragecompressionratioofMPEGis50:1,upto200:1,thecompressionefficiencyisveryhigh,andthequalityofimagesandaudioisalsoverygood,andthereisaunifiedstandardformatonthecomputer,andthecompatibilityisquitegood.

ItisworthnotingherethatDIVX.DIVXvideoencodingtechnologycanbesaidtobeanewvideocompressionformatthatthreatensDVDs.SomepeoplesaythatitisaDVDkiller.ItismodifiedfromMicrosoftmpeg4v3andusesMPEG4compressionalgorithms.Atthesametime,itcanalsobesaidthatitwasdevelopedtobreakthevariousagreementsofASF.AndusingthisencodingtechnologyMPEG4,whichissaidtobeprohibitedfromexportbytheUnitedStates,only2CDROMsareneededtocompressaDVD.ThismeansthatreaderscangetthesamevideoqualityasitdoesnotneedtobuyanadditionalDVDdrive.Andtoplaythiskindofencoding,therequirementsforthemachinearenothigh.TheminimumCPUlimitisonlyrequiredtobeabove300MHZ,andintermsofthechoiceofCPUtype,whetheryourcoreisPII,CELERONorPIII,AMDK6/2,AMDK6III,ATHALON,EvenCYRIXx86canalsobeeatenandwon.Onagraphicscardwith64megabytesofmemoryandan8megabytesofvideomemory,DIVXcanplaysmoothly.

RA/RM/RMVB---RealVideofile

RealVideofileisanewstreamingvideofileformatdevelopedbyRealNetworks.ItisincludedinRealMedia,theaudioandvideocompressionspecificationdevelopedbyRealNetworks,ismainlyusedtotransmitlivevideoimagesinreal-timeonalow-speedwideareanetwork.Differentcompressionratioscanbeusedaccordingtothenetworkdatatransmissionrate,soastorealizethereal-timetransmissionandtransmissionofimagedata.Real-timeplayback.Inadditiontobeingplayedintheformofordinaryvideofiles,RealVideocanalsocooperatewiththeRealServerservertoplayvideoimageswhiledownloadingduringthedatatransmissionprocess.Itdoesnothavetobedownloadedbeforebeingplayedlikemostvideofiles.ManywebsitesontheInternethaveusedRealVideotechnologyforlivebroadcastofmajorevents.

RMVBmovieformatismorethantheoriginalRMVBtwowords,hereVBistheabbreviationofVBR(VariableBitRate-variablebitrate).Onthebasisofensuringtheaveragesamplingrate,themaximumsamplingratevalueisgenerallytwicetheaveragesamplingrate.Itcanalsogetbetterresultswhenprocessingmorecomplexdynamicimages,anditisflexiblewhenprocessinggeneralstillimages.Convertingtoalowersamplingrateeffectivelyreducesthefilesize.

MOV/QT---QuickTimefile

QuickTimeisanaudioandvideofileformatdevelopedbyAppleComputerCompanyforsavingaudioandvideoInformation,withadvancedvideoandaudiofunctions,issupportedbyallmainstreamcomputerplatformsincludingAppleMacOSandMicrosoftWindows95/98/NT.TheQuickTimefileformatsupports25-bitcolor,supportsRLE,JPEGandotherleadingintegratedcompressiontechnologies,providesmorethan150videoeffects,andisequippedwithasounddevicethatprovidesmorethan200MIDIcompatibleaudioandequipment.ThenewversionofQuickTimefurtherexpandstheoriginalfunctionsandincludesthekeyfeaturesofInternet-basedapplications.Itcanprovidereal-timedigitalinformationflow,workflowandfileplaybackfunctionsthroughtheInternet.Inaddition,QuickTimealsoadoptsamethodcalledQuickTimeVR(simplifiedVirtualReality(VR)technologyisusedasQTVRtechnology.Throughtheinteractivecontrolofthemouseorkeyboard,theusercanobservea360-degreescenearoundaplace,orobserveanobjectfromanyangleinspace.QuickTimehasbeenwidelyrecognizedbytheindustryforitsleadingmultimediatechnologyandcross-platformfeatures,smallerstoragespacerequirements,independenceoftechnicaldetails,andhighopennessofthesystem.Ithasbeenwidelyrecognizedbytheindustryandhasbecomethedefactoindustrystandardinthefieldofdigitalmediasoftwaretechnology..TheInternationalOrganizationforStandardization(ISO)selectedtheQuickTimefileformatasaunifieddigitalmediastorageformatforthedevelopmentofMPEG4specifications.

ASF/WMV----MICROSOFTstreamingmediafile

Microsoft’sAdvancedStreamingFormat(ASF,AdvancedStreamingFormat)isalsoaForthetechnicalstandardsofreal-timedisseminationofmultimediaontheInternet,MicrosofthasgreatambitionsandhopestoreplacetechnicalstandardssuchasQuickTimewithASF.ThemainadvantagesofASFinclude:localornetworkplayback,expandablemediatypes,componentdownloads,andscalability.ThemaincomponentsofASFapplicationareNetShowserverandNetShowplayer.AnindependentencodercompilesthemediainformationintoanASFstream,andthensendsittotheNetShowserver,andthentheNetShowserversendstheASFstreamtoallNetShowplayersonthenetwork,soastorealizeunicastormulticast.Thisissimilartothereal-timebroadcastoftheRealsystem.WMVisalsoatechnicalstandardforreal-timedisseminationofmultimediaontheInternetindependentoftheencodingmethod.MicrosofthopestouseittoreplacetechnicalstandardssuchasQuickTimeandfileextensionssuchasWAVandAVI.Themainadvantagesofwmvinclude:localornetworkplayback,scalablemediatypes,componentdownloads,scalablemediatypes,streamprioritization,multi-languagesupport,environmentindependence,richrelationshipsbetweenstreams,andscalability.

AVI(nAVI)

Ifyoufindthattheoriginalplayersuddenlycannotopenavifilesinthisformat,thenyouhavetoconsiderDidyouencounternAVI?nAVIistheabbreviationofnewAVI,anewvideoformatdevelopedbyanundergroundorganizationcalledShadowRealm.ItisamodificationoftheMicrosoftASFcompressionalgorithm(nottheAVIasimagined).Thevideoformatpursuesnothingmorethancompressionrateandimagequality.Therefore,inpursuitofthisgoal,NAVIhasimprovedsomeoftheshortcomingsoftheoriginalASFformat.,SothatNAVIcanhaveahigherframerate.Ofcourse,thisisattheexpenseofthevideostreamingcharacteristicsofASF.Inanutshell,NAVIisanimprovedASFformatthatremovesthecharacteristicsofthevideostream,andcanalsoberegardedasanon-networkversionofASF.

ISO,BIN,IMG,TAO,DAO,CIF,FCD-mirrorfileformat

Themirrorfileisactuallyanindependentfile,andotherfilesDifferent,itismadebymultiplefilesthroughburningsoftwareorimagefilemakingtool.

Thescopeofapplicationofimagefilesisrelativelywide,andthemostcommonapplicationisdatabackup(suchasfloppydisksandCDs).Withthepopularizationofbroadbandnetworks,somedownloadwebsitesalsohavefiledownloadsinISOformat,whichfacilitatestheproductionanddeliveryofsoftwareCDs.CommonimagefileformatsareISO,BIN,IMG,TAO,DAO,CIF,FCD.

YoucanuseWinISOorIsoBustertoopentheimagefile.Therearerelatedtutorialswhereyoudownloadthesesoftware,soIwon’trepeatthem..

Commonprocessingdata

Videoprocessingrequirestheuseoflargeamountsofdataastestandtrainingsets,suchas:VisualObjectClassesChallenge2011(VOC2011)(PASCALVisualObjectClassificationChallenge2011),BEHAVE-Crowds(crowdbehaviorvideodataset),trafficvideodatabase(datafrom2010to2011),IRMarksvideodataset(UniversityofCalifornia,SanDiego,facialmotionvideodatabase),TheHonda/UCSDVideoDatabase(UniversityofCalifornia)SanDiegoUniversity’sFaceTrackingVideoDatabase),VIRATVideoDataset(USDepartmentofDefenseAdvancedResearchProjectsAgencyVIRATvideodatabase),etc.

Informationprocessing

Accordingtotheprincipleofthreeprimarycolors,inthevideofield,themixtureofR,G,andBthreecolorsindifferentproportionsisusedtoexpressacolorfulworld.SincetheoriginalsignalinthecameraandthefinalsignalintheTV,monitor,andcomputermonitorareallRGBsignals,usingRGBsignalsasthetransmissionandrecordingmethodofvideosignalswillundoubtedlyhaveextremelyhighsignalquality.Butthisisnotthecaseinpracticalapplications,because.Ontheonehand,thiswillgreatlyincreasethevideobandwidthandincreasethecostofrelatedequipment;ontheotherhand,itisalsoincompatiblewiththecurrentblack-and-whiteTV.

①Videosignal

⒈YUVcomponentsignal

Accordingtothebrightnessequation,convertRGBtogenerateabrightnesssignalYandtwocolordifferencesignalsU(BY),V((RY),theYUVcomponentsignalisobtained.

CharacteristicsoftheYUVcomponent

BecauseYandUVareseparated,apicturewithonlyYwithoutUandVisblackandwhitegray.Theabilityofblack-and-whiteTVstoreceivecolorTVsignalsistheuseoftheseparationcharacteristicsofYUV.

Thecharacteristicsofthehumaneyecanbeusedtoreducethestoragecapacityrequiredfordigitalcolorimages.Thehumaneye’sabilitytodistinguishcolordetailsisfarmorecomparableTheresolutionofbrightnessdetailsislow.Treatingthedifferentcolorvalues​​ofNadjacentpixelsasthesamecolorvalue(reducingtheresolutionofthecolorcomponents)canreducethestoragecapacitywithoutsignificantlyaffectingtheimagequality.

Y/Csignal

TheUandVsignalsintheYUVsignalarefurthersynthesizedintoachrominancesignalCtoobtaintheY/Csignal.Sincethechrominancesignalisprocessedbyfrequency,thisrecordingmethodItisalsocalledthecolordown-conversionmethod.

Compositesignal

TheYsignalandCsignalintheY/Csignalarefurthersynthesizedtoobtaincompositevideo,whichisafull-colorTVsignal.TheapplicationofcompositevideoismainlyfortheconvenienceoftransmissionandTVsignaltransmission.

②Sampling

③Quantization

④Compression

LossCompression/losslesscompression

Hardwarecompression/softwarecompression

Compressionratio

⑤Computerdisplaysystem

Relatedtechnology

Hardwaretechnology

①Developmentdirectionofvideoprocessinghardware

Combinedwithnetworkcommunicationtechnology,avideocapturecardandanetworkcommunicationcardformamultimediavideoconference,videophone,andvideomail,Multimediacommunicationterminal,etc..

Combinedwithfilmandtelevisionproductiontechnologytoformanon-lineareditingsystemforfilmandtelevisionproductionthatintegratescompression/decompression,compositeoutput,andspecialeffects.

②VideocaptureCard

③Videooutputcard(TVencodingcard)

④MPEG1compression/decompressioncard

⑤TVreceivingcard

⑥NonLineareditingcard

⑦MPEG2compression/decompressioncard

Videoquality

Videoquality(ortranslatedas"imagequality","imagequality")canUseobjectivepeaksignal-to-noiseratio(PSNR)toquantify,oruseexpertobservationtoevaluatesubjectivevideoquality.

●Foravideoprocessingsystem(Forexample,compressionalgorithmortransmissionsystem),atypicalsubjectiveimagequalityevaluationusuallyincludesthefollowingsteps:

●Selectasetofunprocessedvideoclips(calledSRC)asabenchmarkforcomparison.

●SelectSelectthesetvalueoftheprocessingortransmissionsystem(calledHRC).

●Determinethescientificmethodofhowtopresenttheprocessedvideototheevaluatorandcollectitsevaluation.

●Inviteasufficientnumberofevaluators,usuallynolessthan15people.

●Implementassessment.

●CalculatethescoresgivenbyeachevaluatortoeachgroupofdifferentHRCs(usuallyaveraged).

ManymethodsforsubjectiveimagequalityevaluationaredescribedintheITU-TRecommendationBT.500.OneofthestandardizedmethodsisDSIS(DoubleStimulusImpairmentScale).IntheDSISassessment,theevaluatorwillfirstwatchanunprocessedvideoclip,andthenwatchtheprocessedvideoclip.Finally,evaluatetheprocessedvideoclips,rangingfrom"indistinguishablefromtheoriginalimage"to"severelydegradedcomparedtotheoriginalimage."

Videocompressiontechnology

Videocompressiontechnology(onlyapplicabletodigitalsignals).Sincedigitalsignalsystemsarewidelyused,peoplehavedevelopedmanymethodstocompressvideostreams.Sincevideodatacontainsspatialandtemporalredundancy,makinguncompressedvideostreamsisquitebadfromthepointofviewoftransmissionefficiency.

Ingeneral,spatialredundancycanbereducedby"onlyrecordingthedifferencebetweenonepartofasingleframeandanotherpart";thistechniqueiscalledintra-framecompression(intraframecompression).Anditiscloselyrelatedtoimagecompression.Thetemporalredundancycanbereducedby"onlyrecordingthedifferencebetweentwoframesofdifferentpictures";thistechniqueiscalledinterframecompressionandincludesmotioncompensationandothertechniques.Atpresent,themostcommonlyusedvideocompressiontechnologiesareMPEG-2,whichisusedinDVDandsatelliteliveTV,andMPEG-4,whichiscommonlyusedinInternettransmission.

Bittransferrate

Bittransferrate(onlyapplicabletodigitalsignals),bittransferrate(alsotranslatedasbitrateorbitrateorbitrate)isoneAwaytoexpresstheamountofinformationcontainedinavideostream.Itsunitofquantityisbit/s(thenumberofbitstransmittedpersecond,alsowrittenasbps)orMbit/s(thenumberofmillionbitstransmittedpersecond,alsowrittenasMbps).Ahigherbittransferratewillaccommodatehighervideoquality.Forexample,thepicturequalityofDVDformatvideo(typicalbittransferrateis5Mbps)ishigherthanVCDformatvideo(typicalbittransferrateis1Mbps).TheHDTVformathasahigher(approximately20Mbps)bittransferrate,andthereforehasahigherpicturequalitythanDVD.

Variablebitrate(VBRinabbreviation)isameansofpursuingtheimprovementofvideoqualityandreducingthebittransmissionrateatthesametime.VBR-encodedvideowillautomaticallyrecordimagesatahigherrateduringlargedynamicorcomplexpictureperiods,butwillreducetherateduringstaticorsimplepictureperiods.Inthisway,thetransmissionratecanbereducedasmuchaspossiblewhileensuringaconstantpicturequality.Butforvideostreamsthathaveafixedtransmissionbandwidth,requireimmediatetransmissionandnotemporarystoragemeans,aconstantbitrate(CBR)ismoresuitablethanVBR.Videoconferencesystemisanexample.

Impact

Mostconsumervideoproductsaredistinguishedbythevideoprocessingtechnologiesandalgorithmsusedduringandafterdecompression.Somevideoprocessingtechnologiesmaybeimplementedindifferentwaysthanthoselistedbelow,andthefollowingtechnologiesmayhavedifferentnamesindifferentapplications.

MoviesandTV

Moviesareusuallyrecordedatarateof24framespersecond.TowatchonTV,youneedtoconvertthevideotohowmanyfieldspersecond.Thisprocessiscalledtelecine.),orX:Yfolding.ConvertingmotionpicturefilmtoNTSCtelevisionsystemiscalled3:2folding,whichconverts24framespersecondtoapproximately60fieldspersecond.Thisconversionprocessconsistsoftwosteps:First,therateofthemovieisreducedby1%.Secondly,the4framesinthemovieareconvertedinto10-fieldNTSCformat.ThisconversionprocessisrealizedbyusingtheinterleavingnatureofNTSCformattoextend4framesto5frames.AsaTVmovie,3:2(or2:3actually)meansthatthefirstframeofthemoviedisplays2fields,thenthesecondframedisplays3fields,andthethirdframedisplays2fields,andsoon.Duringanyfilm-to-TVconversionprocess,therewillbesomedistortioncalled"jitter".Inordertoeliminatethisjitterandregeneratetheoriginalsignal,itisnecessarytoadoptareversefilmandtelevision(reverse3:2folding)methodtoconverttheconvertedsignalbackto24framespersecond.Reversingfilmandtelevisionisawayofde-interlacing.

De-interlacing

InordertotakefulladvantageofHDTVsandotherdisplaysthatdisplaynon-interlaced(orprogressivescan)video,thevideobroadcaststreammustbeconvertedfrominterlacedtonon-interlaced,Thisconversionprocessiscalledde-interleaving.Itisnecessarytoperformde-interleavingwhenconvertingcompositesignalssuchasNTSCintocomponentsignalssuchasRGBorYCbCr.Thereareusuallyseveraldifferentmethodsofde-interleavingalgorithms(withdifferentnames),includingfilmmode,videomode,motioncompensation,scenecombining(interleaving,mixing,selectivemixing)andfieldexpansion(halfsize,two-wiretechnology),etc..Eachmethodhasitsownadvantagesanddisadvantages,andthebestde-interleavingmethodistocombinethesemethods.MPEGdecoderssometimesstilluse3:2folding,sothereisnoneedtousede-interleaving.

Scaling

Scalingisalmostnecessaryforeveryvideodevicethatsupportsmultipleresolutions.ThisisbecausethesignalmustbeconvertedfromoneresolutiontoasuitableAnotherresolutionoftheterminaldevice.Whenusingde-interleaving,thecomponentsignalscannotbescaledunlesstheyareconvertedtocompositesignalsfirst.Thereareseveralscalingschemes,images,andpixelcopyingistoenlargetheimagebycopyingpixels.Asyoucanimagine,thismethodcreatesmarginandintroducesdistortion.Linearinterpolationisnotmuchbetter,itcanalsocausedistortion,especiallyforhigh-frequencycomponents.Theeffectofanti-aliasingre-samplingisthemostidealbecauseitcanensurethatthefrequencycomponentsarecorrectlycalibrated.Thereareseveralanti-aliasingmethods,andthebestmethodneedstobedeterminedaccordingtotheterminaldevice.

Deblockingandderinging

Videoframesareusuallydividedintogroupsof16samplesx16lines.Suchgroupsarealsocalledmacroblocks.ThesemacroblocksareusedforMotionestimationandmotionvector.Whenmacroblocksaredecompressedandreproduced,theedgesofadjacentmacroblocksmaynotparticularlymatch,becausenaturalerrorsoccurwhenreconstructingthedamagedcodec.Whentheedgesarenotwellmatched,theboundariesofthemacroblocksareexposed.Inordertoeliminatetheseundesirableconsequences,alow-passdeblockingfiltermustbeusedtomixandeliminatethesedistortions.H.26?containsadeblockingfilterappliedtothemacroblocklevel,whichmustbeconstructedseparatelyforothercodecs.Whentheencoderlosestoomuchinformationduringthemacroblockquantizationprocess,theimageedgeswillbedistorted.Thistypeoferroriscalledringing.Justlikedeblocking,deringingrequiresanadaptivelow-passfiniteimpulseresponsefiltertomasktheringingeffect.

Othertechnologies

Theremaybemanyreasonsforerrorsduringbroadcastorplayback.Forexample,DVDsareoftenscratchedandcannotbecorrected.Errorconcealmentmethodscanoftenmakeupforuncorrectableerrors.Techniquessuchasinterpolationorreplacingcorrupteddatawithpreviousandsubsequentframedatacanbeusedtohidethistypeoferror.Theedgeenhancementmethodusesaprocessfiltertoincreasethecontrastofthebrighteranddarkerpixelsonbothsidesoftheedgetoimprovethesharpnessoftheimage.Thisprocessingtechnologyismoreusefulforlow-enddisplays,anditisbetterforlong-distanceviewing.Becausetheedgeenhancementmethodreducesthequalityoftheimage,itisgenerallynotusedforhigh-enddisplays.Transcodingisthedirectconversionfromonedigitalsignalformattoanotherformat,whichinvolvesselectinganencodedvideoformat,decodingordecompressingitintotheoriginalformat,justlikereplayingit(Figure1).Then,thevideostreamiscompressedorencodedintoanewdesiredformat.Mostcodecsarelossy,andtheerrorsanddistortionscausedwillaccumulate,eventuallyleadingtocontinuousdeteriorationofimagequality."Transcodingwillbecomeoneofthemostchallengingfunctionsofvideosystemsinthenextfewyears,"saidJeremiahGolston,ChiefTechnologyOfficerofTI'sStreamingMediaDivision."Whenconvertingbetweenavarietyofdifferentvideocodecformats,bitratesandresolutions,high-qualityvideosignalsmustbemaintainedtomeettheneedsofhomemediadevicestosharecontentwitheachother."Golstonadded."InthecaseofverytightsystembudgetssuchasI/Obandwidthandoverallcost,itisnecessarytoinnovateinalgorithmsandarchitecturestoachievethesegoals.Real-timeHDtranscodingsolutionsbasedonDSPtechnologycanprovidebothhighperformanceandflexibility,Soastomeetthesedemandingrequirements."DedicatedcodecandvideoprocessingchipIfspeedisthepriority,adedicatedchipwithcodecandvideoprocessingfunctionswillbethefirstchoice.Table1listsaseriesofexistingchipsthathavecodecandimageprocessingfunctions.Thetabledoesnotincludeallrelatedproducts,andmostofthecompanieslistedprovidemorethanoneoraseriesofvideoprocessingproducts.High-definitionTVismorethanjustdigitalprocessing.Thesinglefixeddisplayresolutionofaflat-panelHDTVmustmatchmultipledisplaystandards,sothereisadesignproblemthatCRTTVshavenotexperienced.TherearemanywaystosolvethisproblemforthegraphicscardinthePC,buttheHDTVreceivermustItisfasterandbetterthanaPCgraphicscard,andthechallengeisevenmoreseverewhenHDTVmusthandletraditionalvideofromVHS/DVDplayers.However,thisproblemismainlyforhigh-endsystems.Mass-marketHDdevicesonlyneedtodealwiththecableinputproblem.InordertoobtainanexternalSDdatasource,consumersmustpayforadditionalconnectors.Analogfront-end(AFE)chipscancompletethisprocessing,buttheyaremuchmorecomplicatedthanthewell-knownAFEs,whichareonlyADCsignalconditioners.Somesimulationcompaniesprovideproductsthatmanagetosolvetheseproblems.Intersil'sautomaticblacklevelcompensation(ABLC)functionandAnalogDevices(ADI)videonoiseshapingtechnologysolvethesechallengesinauniqueway.Theanalogvideosignalcontainsahorizontalandverticalretraceinterval.Duringthisinterval,theelectronbeamoftheCRThastoberedirectedtothebeginningofanewlineorfield,butbecauseflat-panelTVslackverticalorhorizontalretrace,theyneedonepixel.clock.Theanalogvideosignaldoesnotprovideapixelclock,soaphase-lockedloop(PLL)isrequiredtogeneratetheclock.HDresolutionrequiresalow-jitterPLL,buttherangeallowedbytheHDstandardposesadesignchallengeforanalogPLLs,becausetheloopfilterisdifficulttooptimizewhenthehorizontalcomponentfrequencyis10-150kHz.OffsetisanotherinherentAFEchallenge.TheAFEvideopathusuallyincludesaDCrestorationclampcircuit,offsetandgaincorrection,andananalog-to-digitalconversioncircuit.AgoodDCrecoverycircuitcaneliminatetheoffsetattheinputoftheAFE,butsubsequentactivedeviceswillalsogenerateoffsets.Theoffsetisrandomandwillvaryfromdevicetodevice.Theyusuallyhavealargetemperaturecoefficient,sotheoffsetwilldriftwhenthedisplayheatsup.Herearetheissuesrelatedtooffset.Inthecomponentvideosignal,Ysignal(luminance,graylevelinformation)andPb,Prsignal(chroma,colorinformation)aresentoutthroughthreeindependentchannels.TheYsignalisunipolar,andtheYoffsetontheYchannelwillaffectthebrightness.ThePbandPrsignalsarebipolar,andtheyformanorthogonalcolorspace.TherandomoffsetofPbandPrwillmovethecenterofthespacefrom0volts,whichwilladdcolortotheimagethatshouldbegray,therebymovingtheentirecolorspace,resultinginincorrectcolordisplay.Inthepast,displaymanufacturersdidonlyonecalibrationordidnotdealwiththisproblematallduringdisplayproducttesting,andthedeliveredequipmenthadlargeblacklevelsandcolorvariations.Theresultofthisisthattheuserneedstomanuallyadjustthesettingvalue.

Processor

Mainstreamvideoprocessor

Inordertocreateasuccessfuldigitalvideoproduct,youneedtochoosetherightprocessor.Itsoundssimple,ofcourse,itisnot.Oneofthebigproblemsisthattherearetoomanyprocessorstylesforyoutochoosefrom:general-purposeCPUs,FPGAs,DSPs,configurableprocessors,fixed-functionchips,andothertypesofprocessors.

Whatmakestheproblemmorecomplicatedisthatdigitalvideoisarapidlydevelopingfield,andstandardsareconstantlyevolvingandchanging.Therefore,inthefieldofdigitalvideo,theabilityoftheprocessortoadapttochangesismoreimportantthaninotherapplications,butthisflexibilityusuallycomesatthecostofreducedefficiency.

Whenchoosingaprocessor,itisinevitabletomakesomecompromises,butthekeyistoknowhowtochooseaprocessorsothatthesuccessoftheproductwillnotbecompromised.

Takewhatyouneed

Becausetherearesomanyprocessorsforyoutochoosefrom,it’sactuallyimpossibletoreadthemall—youcan’tevenlearnaboutthemaincategoriesindetail,soyoucanuseGradingmethod:Usethemostimportantscreeningcriteriatoeliminateunsuitablecandidatesfirst.

Thecommonlyusedcriteriafortheprimaryselectioninclude:

*Speed.Digitalvideotasks,likemanyothertypesofsignalprocessingtasks,requiretheprocessortoundertakeheavycomputingtasks.Forthetargetapplication,carefullyanalyzewhethertheprocessorhassufficientspeed,anditisbesttousevideo-orientedtestbenchmarkssuchasBDTIVideoBenchmarks.

*Price.Althoughchippriceisimportant,costperchanneloroverallsystemcostmaybemoreimportant.

*Energyefficiency.Inmostcases,evaluatingenergyefficiencyismoremeaningfulthanpowerconsumption,becauseenergyusagedeterminesbatterylife.

*Flexibility.Sometypesofprocessorsaremoreflexiblethanothertypesofprocessors,canadapttofutureproductcharacteristicschanges,orallowfieldupgrades,suchasaddingsupportfornewcompressionalgorithms.However,ingeneral,themoreflexibletheprocessor,theloweritscostandenergyefficiency.

*Thequalityofdevelopmenttools.Whethertheprocessorhasthetoolstosupportthedevelopmentofsignalprocessingapplications(orfurthermore,videoapplications)mayhaveasignificantimpactonthedevelopmenttimeand,inturn,thetimetomarket.

*Compatibilitywithearlierprocessormodels.Thisisusuallyimportantifyouexpecttoreusesoftwareusedinearlierproducts.

*Supplierroadmap.Doesthesupplier'sproductroadmapfityourfollow-upproductdevelopmentplanverywell?Willtheprocessorbesupportedorupgradedduringthelifeofyourproduct?

*Soldintheformofchipsorlicensablekernels.Someprocessorsaresoldaspackagedoff-the-shelfchips;somearesoldaslicensableintellectualproperty-oftencalledlicensablecores,usedtomakecustomchips.Mostoftheprocessortypesdiscussedinthisarticleincludebothpackagedchipsandlicensablecores.

Prosandconsofvariousprocessors

Focusonthesixtypesofprocessorscommonlyusedindigitalvideo:fixedfunctionengines,application-specificstandardproducts(ASSP),Mediaprocessor,DSP,embeddedRISCprocessorandFPGA.Theserangescoverthemostprofessionalandflexibleproducts,discusstheprosandconsofvarioustypesofprocessors,andanalyzeaspecificproductineachcategory.

Firsttalkaboutthefixedfunctionengine.Itusesahard-wiredprocessorstructureformaximumefficiency;theydonotuseinstructionstreamsandcannotbeprogrammed.Hard-wiredlogicsacrificesflexibilityinexchangeforextraordinaryprocessingspeed,energyefficiency,andoftencost-effectiveness.

Usingafixed-functionenginecansimplifysystemdesignandtesting.Sincethefixedfunctionengineisnotprogrammable,productdevelopersdonothavetolearnprogrammingtoolsorintegratemultiplesoftwaremodules.Andtheydonotneedtoconsiderwhetherthemultipletasksperformedbytheprocessormayaffecteachotherinanundesirableway,orwhetheritwillinterferewiththereal-timecharacteristicsofthesystem.

Fixedfunctionenginesaregenerallyprovidedintheformoflicensableintellectualproperty(IP)tofacilitateintegrationintocustomchips.Inthisform,fixed-functionenginesaremostsuitableforhigh-volumeapplicationssuchasmobilephones.Fixed-functionenginesaresometimesalsoavailableintheformofchips.Fixed-functionvideochips,suchasMPEG-2decoderchips,canaddfunctionstoexistingproductsatalowercost,especiallywhentheproducthasahostprocessorthatcanhandledemandcontrolanduserinterfacefunctions.

Forexample,Hantro's5150MPEG-4videodecoder,whichisanexampleofafixed-functionenginesoldintheformofIP.Theengineisintendedtobeusedasaco-processor,attachedtoageneral-purposeprocessor,whichhandlessomelessdemandingsubtasksrequiredforMPEG-4decoding.

Themaindisadvantageoffixed-functionhardwareisitslackofflexibility.Becauseitisnotprogrammable,productdeveloperscannoteasilymodifyfixed-functionhardwaretosupportnewstandardsordifferentfunctions.Thisisacauseforconcern,becausemanyvideoapplicationsarestillimmatureandthestandardsandfunctionsarechangingrapidly.

Therefore,fixedfunctionenginesareoftenusedaspartofdedicatedstandardproducts.Let'stalkaboutASSP.

DedicatedStandardProducts(ASSP)arededicatedchipswithhighintegration.Itcanbecomparedwithanapplication-specificintegratedcircuit(ASIC).ASICsaredesignedbysystemcompaniesandusedintheirownproducts.TheASSPisdesignedbyachipcompanyandprovidedtomultiplesystemdevelopersasaready-madechip.Becauseitisexpensiveandtime-consumingtodevelopacomplexchip,ASSPisusuallyusedforproductsthathavereachedalargenumberoffinalizedproducts,orareexpectedtohaveahighervolumeofproducts.

Forexample,Zoran’sVaddis5RisahighlyspecializedchipforaudioandvideoprocessinginDVDrecorders.Thekeyalgorithmsrequiredhavebeendetermined:themostprominentisMPEG-2videocompressionanddecompression.

AlthoughVaddis5RcontainstwoRISCprocessors,itusesfixed-functionhardwareacceleratorswhenperformingthemostcomputationallyintensivetasks,suchasMPEG-2videodecodingandcolorspaceconversion.Fortheabovereasons,Vaddis5R(andothersimilarASSPs)alsohavetheadvantagesanddisadvantagesoffixedfunctionengines:goodperformanceandenergyefficiency,butlimitedflexibility.

Limitedflexibilitymeansthatwhensystemdesignersdesignproducts,theycaneasilymaketheirproductssimilartootherproductsusingthesameASSP.Italsomeansthatsystemdesignersarehighlydependentonthechipsupplier’sroadmapbecausenewchipsarerequiredtosupportsignificantlydifferentfunctions.

ThereisalsoanASSPthatreliesmainlyonprogrammableprocessorstoperformheavyvideotasks,andsacrificesenergyefficiencyandcostefficiencyforflexibility.ThistypeofASSPisusuallybundledwithkeysoftwaresuchasvideodecodersandhardwaredrivers,sothatsystemdevelopersdonothavetoengageinmanylow-levelsoftwaredevelopmentwork.However,comparedwiththeuseofASSPbasedonfixed-functionhardware,itmaystillrequirealotofeffortinsoftwaredevelopmentandintegration.

ThemediaprocessorisbetweentheASSPandthedigitalsignalprocessor(DSP)intermsofspecialization/flexibility.Themediaprocessorisoptimizedfortasksrelatedtoaudioandvideoprocessing,andisnotoptimizedforawiderangeofsignalprocessingtaskslikeDSP.Mediaprocessorsusuallyusemultipleprocessors,includingamainprocessingenginesimilartoaDSP,twoorthreededicatedcoprocessors,andaudioandvideodedicatedperipherals.

PhilipsPNX1500isamediaprocessor.Likeatypicalmediaprocessor,PNX1500isbasedonapowerful,highlyparallelprocessorcorethatishighlyefficientwhenperformingvideoprocessingtasks.PNX1500alsocontainsseveralfixed-functionhardwareacceleratorsanddedicatedperipheraldevices,whicharealsothecharacteristicsofatypicalmediaprocessor.Themainprocessorcorehandlescomplexvideotaskssuchascompression,andsystemdesignerscanprogramthecore.

SimilartoZoran’sVaddis5R,PNX1500isverysuitableforMPEG-2decoding.ButunlikeZoran’sASSP,PNX1500isflexibleenoughtobeusedwithothervideocompressionstandardssuchasH.264.Ofcourse,thisflexibilitycomesataprice:comparedtofixed-functionhardware,softwarevideodecodersaregenerallylessenergyandcostefficient.

Becausethemediaprocessorusesmultipledifferentprocessors,comparedwithotherprogrammableprocessors,itcausesgreatdifficultiesinsoftwaredevelopment.Forexample,inordertoperformaspecificvideotask,itisgenerallynecessarytoprogramtwoormoreprocessingunitsandcoordinatethem.Tohelpmakeupforthisshortcoming,mediaprocessorvendorsoftenprovideoptimizedsoftwarecomponentlibraries.

MediaprocessorvendorsusuallyemphasizetheuseofCorC++whendevelopingsoftware,anddonotrecommendorsupportassemblylanguage.Thereasonforemphasizingtheuseofhigh-levellanguagetodevelopsoftwareistopreventprogrammersfromhavingtofacemanycomplexissuesinprocessorarchitecture.Butthedownsideisthattheprogrammermustrelyonthecompilertogeneratevalidcode,whichisnotalwaysrealistic.Developersmayneedtoinvestalotofefforttomanuallyadjusttheirhigh-levellanguagecodetogetthebestperformance.

Digitalsignalprocessor(DSP)isdesignedforaseriesofsignalprocessingapplications.Comparedwiththemediaprocessor,thespecificationsadoptedbytheDSParelessfocusedonvideoprocessing,andtheparallelismislow.Inordertomakeupfortheweaknessoflowparallelism,inagivenapplication,theinstructionexecutionspeedoftheDSPmustusuallybehigherthanthatofthemediaprocessor.Highercommandspeedmaycomplicatesystemdesignandincreaseenergyconsumption.Ontheotherhand,comparedwiththeembeddedRISCprocessor(describedbelow),theDSPrequiresalowerclockspeedwhenprocessingvideotasks.ThekeyadvantageofDSPliesinitsflexibilityandpowerfulapplicationdevelopmenttools.