Planning the Ver i Plan

7/27/2019 Planning the Ver i Plan

1/44

[email protected]

www.verificationacademy.com

Verif ication Planning and

Management Introduction

Plan of Attack

Peet JamesSenior Verification Consultant


2/44

Verification Effort Summary

Why Plan? Because verification takes more time & effort than design Because it takes a separate, informed, documented

architecture, process and team Because ad-hoc verification will lead to minimal results and

much wasted effort

Why is it Hard? Because there is too much to do Because people resist change

Because it more than just tools & training, it also takesdeployment Because complex change takes time and happens in stages Because it is a steep learning curve


3/44


Its about: Not re-inventing the wheel Making informed, strategic decisions early Avoiding pitfalls, and having to re-coding Finding and fixing problems early Coding for debug

Choosing your verification battles Setting up for successful feedback & guidance Ramping up quickly

Engineering is about: Managing Change

Organizing large quantities of data Defining & clarifying processes & problems Downsizing larger problems into smaller, doable problems


4/44


Its about: Not re-inventing the wheel Making informed, strategic decisions early Avoiding pitfalls, and having to re-coding Finding and fixing problems early Coding for debug

Choosing your verification battles Setting up for successful feedback & guidance Ramping up quickly

Engineering is about: Managing Change

Organizing large quantities of data Defining & clarifying processes & problems Downsizing larger problems into smaller, doable problems

Bottomline: We need a PLAN OF ATTACK toverifying the most in the allotted time


5/44

Verif ication Plan of Attack

Plan

Architecting averification

solution

30,000 ft to1,000 ft view

Big Picture


6/44


Plan Populate


solution


Big Picture Details

Requirements

1,000 ft to1 ft view


7/44


Plan Populate Pilot


solution


Big Picture Details

Requirements


Day to Day

ManagingThe entireVerification

SystemStart

To finish


8/44

Verification Planning: PLAN

Big Picture

Mission & Goals

Strategizing

Architecting

Layers

Phases

Q&A

What and How and

in What order wewill build the

testbench?

Plan


solution


Big Picture


9/44


Big Picture

Mission & Goals

Strategizing

Architecting

Layers Phases

Q&A

What and How and

in What order wewill build the

testbench?

Main Areas of Focus

Generation

Checking:

Scoreboard,

Assertions Coverage

Automation

Hardest part to

automate

ReUse

Libraries: AVM, OVM

Plan


solution


Big Picture


10/44


Output: VAD = Verification Architect Document

VIDs = Verification Implementation Docs

VI = Verification Infrastructure

Deliverables: Basic Strategy / Approach

Basic Structure Block Diagram

List of Components Line & Effort Estimates

that can lead to a schedule

Basic OOP setup: UMLs & Layers Basic Build Order Phases & Milestones

Basic ReUse

Plan

VAD /VID

VerificationInfrastructure


solution


Big Picture


11/44

Verif ication Planning: POPULATE

Requirements Extract

Refine

Prioritize

Track

Q&A: What & Whenspecifically will we?

Apply to the DUT

Check in the

scoreboard

Check wi th

assertions

Cover

Populate

DetailsRequirements



12/44


Requirements Extract

Refine

Prioritize

Track

Q&A: What & Whenspecifically will we?

Apply to the DUT

Check in the

scoreboard

Check wi th

assertions

Cover

Main Areas of Focus Generation

Checking:

Scoreboard,

Assertions

Coverage

Automation:

Grow your own:

ACCESS, SQL,

QuestaVerificationMgmt with UCDB

ReqTracer

Populate

DetailsRequirements



13/44


Populate

DetailsRequirements


Extraction / Priorit izing / Implement

DR Prioritization

ImplementationVR Translation

DR Gen/Chk/CovVR

VR Mapping:VR VI

DR Extraction

DR

VR Verification Requirement

Design Requirement


14/44


Populate

DetailsRequirements



DR Prioritization


DR Gen/Chk/CovVR

VR Mapping:

VR VI

DR Extraction

DR

DR

DR

DR

DR

DR

DR

DR


Design Requirement


15/44


Populate

DetailsRequirements



DR Prioritization


DR Gen/Chk/CovVR

VR Mapping:

VR VI

DR Extraction

DR

DR

DR

DR

DR

DR

DR

VRVR VR

Checking:

SB / AssertGeneration

VRVR

VR

Cover

VR

VRDR


Design Requirement


16/44


Outputs

Database: xls, UCDB, etc.

Reports

Deliverables: General

List of Generation Items Sequences /Scenarios

List of Scoreboard Elements

List of Assertions Areas

List of Coverage Areas

Populate

DetailsRequirements


DetailedRequirements

CustomDocs


17/44

Verif ication Planning: PILOT

Day to DayManagement

Document

Run & Control

Analyze & Debug

Measure

Q&A: How do we? Control & keep track

of everything

Repeat the results? Know how we are

doing?

Know we are done?

Pilot

Day to DayManaging

The entireVerification

SystemStart

To finish


18/44


Day to DayManagement

Document

Run & Control

Analyze & Debug

Measure

Q&A: How do we? Control & keep track

of everything

Repeat the results? Know how we are

doing?

Know we are done?

Main Areas ofFocus

Staff & Schedule

Rules of Engagement

Run Management

Triage

Metrics

Automation Templates &

Conventions

Scripts

Tools: Bugzilla, VRM,

SVN, etc/

Pilot

Day to DayManaging


SystemStart

To finish


19/44


Outputs

Docs: File Structure, Templates, Conventions

Auto Docs: Schedule, Status Reports

Script Code

HDL, HVL Code

Deliverables

Repeatability

Regression Management

Change & Release Management Data: Gather, Measure, Respond

Pilot

Day to DayManaging


SystemStart

To finish

DocsCode

Scripts


20/44


Plan Populate Pilot

VAD /VID



solution


Big Picture DetailsRequirements


Day to DayManaging


SystemStart

To finish


CustomDocs

DocsCode

Scripts

Verification Architecture & Implementation Docs


21/44

Verification Flows

START END

Plan Populate Pilot


22/44

Verification Flows

START END

START END

Plan Populate Pilot

PlanPopulate Pilot


23/44

Verification Flows

START END

START END

START END

Plan Populate Pilot

Plan

Plan

Populate Pilot

Populate

Pilot


24/44

Verification Flows

START

END

Plan

Populate

Pilot


25/44

Verif ication Plan of Attack: Methodologies

Plan Populate Pilot

VAD /VID



solution




Day to DayManaging


SystemStart

To finish


CustomDocs

DocsCode

Scripts


26/44


Plan Populate Pilot

VAD /VID



solution




Day to DayManaging


SystemStart

To finish


CustomDocs

DocsCode

Scripts

DDV=DocumentDriven

Verification


27/44


Plan Populate Pilot

VAD /VID



solution




Day to DayManaging


SystemStart

To finish


CustomDocs

DocsCode

Scripts

RDV=RequirementDriven

Verification

DDV=DocumentDriven

Verification


28/44


Plan Populate Pilot

VAD /VID



solution




Day to DayManaging


SystemStart

To finish


CustomDocs

DocsCode

Scripts

DDV=DocumentDriven

Verification


Verification

CDV=Coverage Driven VerificationVerif Manager, UCDB


29/44


Plan Populate Pilot

VAD /VID



solution




Day to DayManaging


SystemStart

To finish


CustomDocs

DocsCode

Scripts

DDV=DocumentDriven

Verification


Verification

VerifRunManager

TriageAnalylsis

M

DV=MetricDriven

Verification



30/44

Verification Plan of Attack:

Methodologies



Plan Populate Pilot

VAD /VID


solution




Day to DayManaging


System

StartTo finish

CustomDocs

DocsCode

Scripts

DDV=DocumentDriven

Verification


Verification


VerifRunManager

TriageAnalylsis

M

DV=MetricDriven

Verification


31/44

VPM Outline

Plan Populate Pilot

Discussions

Audit

Strategize

Architect

VAD -Verification

Architecture Doc

VID VerificationImplementation Doc

Verification

Infrastructure (VI)

DDV=Document Driven Verification

RDV=Requirement Driven Verification

MDV=Metri c Driven Verif ication

CDV=Metric Driven Verif ication


32/44

VPM Outline

Plan Populate Pilot

Discussions

Audit

Strategize

Architect

VAD -Verification

Architecture Doc


Verification

Infrastructure (VI)

Discussions

Requirements:

Hunt, Refine,

Prioritize, Track

Design

Requirements Verification

Requirements

Translation &

Mapping






33/44

VPM Outline

Plan Populate Pilot

Discussions

Audit

Strategize

Architect

VAD -Verification

Architecture Doc


Verification

Infrastructure (VI)

Discussions

Requirements:

Hunt, Refine,

Prioritize, Track

Design

Requirements Verification

Requirements

Translation &

Mapping

Discussions

Staffing &

Scheduling

Verification Run

Manager

Triage & BugTracking

Change Mgmt

Metrics &

Dashboards






34/44

VPM Deployment Paths - VPMlite

Plan Populate Pilot

Why Plan?Why Verification is hard?How to do a audit?How to make a VAD?

VAD Example

Approach Hit highlights only

Introduction only

Al l lecture, no application

Half day to one day

Requirements huntingoverview

Requirementsaproaches

Requirements example

Scheduling overviewStaffing overviewMetric overview


35/44


Plan Populate Pilot


VAD Example


Introduction only


Half day to one day

Pros Quick

Cheap

Great intro for further VPMdeployment later

Good for management

Cons Minimal know ledge transfer

No direct application

Minimal retention






36/44


Plan Populate Pilot


VAD Example


Introduction only


Half day to one day

Pros Quick

Cheap

Great intro for further VPMdeployment later

Good for management

Cons Minimal know ledge transfer

No direct application

Minimal retention





Only Recommended as ateaser to doing a future,

real VPM


37/44


38/44

VPM Deployment Paths - VPMfull

Approach

Deployment through out the entireproject: start to finish

Most lectures & topics

Fully customized

Timeframe: Months

Pros Full verification planning documents:

VAD, VIDs & requirements database

Maximum knowledge transfer & retention

Maximum application to the project

Cons: Expense

DDV

Plan:

Decide & Document

Build:

Write & Debug VI

Run:

Find & Fix RTL BugsPlan

Populate

Pilot

Cover:

Write, Analyze & SteerVAD VIDs

VI SV

FilesTBAI

CoVI BlockDiagram

mponent ListLayers & Phases

Schedule

SeparateConsulting

Modules:

TBAI=OVMTestbench

ABV =Assert ion

Hunt, Extract, Refine & Priorit ize

Translate DRs to VRs & map to VI

RDV:RQMTs xls

RDV:

Traceability: Schema & Pipeline

Link: To co verage & assertions

RQMTs Database

CDV: UCDBStaff & ScheduleRules of EngagementRun & Triage StrategyMetric Strategy

ABV

Verif Manager, VRM, Triage & MDV

1 3 Weeks 1 3 Months Months Months


39/44

VPM Deployment Paths - VPMfull

Approach

Deployment through out the entireproject: start to finish

Most lectures & topics

Fully customized

Timeframe: Months

Pros Full verification planning documents:

VAD, VIDs & requirements database

Maximum knowledge transfer & retention

Maximum application to the project

Cons: Expense

DDV

Plan:

Decide & Document

Build:

Write & Debug VI

Run:

Find & Fix RTL BugsPlan

Populate

Pilot

Cover:

Write, Analyze & SteerVAD VIDs

VI SV

FilesTBAI

CoVI BlockDiagram

mponent ListLayers & Phases

Schedule

SeparateConsulting

Modules:

TBAI=OVMTestbench

ABV =Assert ion

Hunt, Extract, Refine & Priorit ize

Translate DRs to VRs & map to VI

RDV:RQMTs xls

RDV:

Traceability: Schema & Pipeline

Link: To co verage & assertions

RQMTs Database

CDV: UCDBStaff & ScheduleRules of EngagementRun & Triage StrategyMetric Strategy

ABV

Verif Manager, VRM, Triage & MDV

Recommended for maximumstrategic change

1 3 Weeks 1 3 Months Months Months


40/44

VPM Deployment Paths VPMtypical

Approach

Deployment at the start of aproject & then at key times

Key lectures & topics

Customize for Low HangingFruit, best bang for the buck

Timeframe: Month Overall

DDV

Plan:Decide & Document

Build:Write & Debug VI

Run :Find & Fix RTL BugsPlan

Populate

Pilot

Cover:Write, Analyze &

SteerVAD VIDs

TBAIVI Block DiagramComponent ListLayers & Phases

Schedule

SeparateConsulting

Modules:

TBAI=OVMTestbench

ABV =Asser tion

Pick DRs or VRs only

Example subset Extracted

RDV: RQMTs xls

CDV:

Staff & SchedulePick one issueVRM, Triage, MDV, etc

ABV

MDV

3 Days 2 Weeks The Rest of the Project


41/44


Approach





Pros

Strategic Verification planning todrive sane deployment

Plot for key knowledge transfer &retention

Plot for st rategic application to theproject

Cons: Team left to self-guidance on somethings

DDV




Populate

Pilot


SteerVAD VIDs


Schedule

SeparateConsulting

Modules:

TBAI=OVMTestbench

ABV =Asser tion



RDV: RQMTs xls

CDV:


ABV

MDV



42/44


Approach





Pros

Strategic Verification planning todrive sane deployment

Plot for key knowledge transfer &retention

Plot for st rategic application to theproject

Cons: Team left to self-guidance on somethings

DDV




Populate

Pilot


SteerVAD VIDs


Schedule

SeparateConsulting

Modules:

TBAI=OVMTestbench

ABV =Asser tion



RDV: RQMTs xls

CDV:


ABV

MDV

Best Overall Trade Off



43/44

Plan of Attack Summary

Customize your path between typical & full Plan:

- Start with Planning & DDV up front:

o An hour of up front planning will save days of rework later

- Make a VAD & let it drive the building of a solid VI in logical layers

and phases

Populate:- Choose your RDV path over several projects

o Choose either DRs only, VRs only or both

- Choose your CDV integration

Pilot:- Choose which piloting battles over several projects

o Rules of Engagement, VRM, Triage, MDV


44/44

[email protected]

www.verificationacademy.com

Verif ication Planning and

Management Introduction

Plan of Attack

Peet JamesSenior Verification Consultant

Planning the Ver i Plan

Documents

Transcript of Planning the Ver i Plan