A Gentle Introduction to F*

Schedule today

  • Cédric: Overview of Project Everest

  • Cătălin: A Gentle Introduction to F*:
    Verifying purely functional programs

  • Cătălin: Verifying Stateful Programs in F*

  • Cédric: Verifying crypto algorithms

  • Exercise session

Please ask questions any time!

Program verification: Shall the twain ever meet?

Interactive proof assistants Semi-automated verifiers of imperative programs
Coq, CompCert,   air Dafny, Verve,
Isabelle, seL4, FramaC, IronClad,
Agda, Bedrock, Why3 miTLS
Lean, 4 colors   gap Vale
  • In the left corner: Very expressive dependently-typed logics,
    but only purely functional programming

  • In the right: effectful programming, SMT-based automation,
    but only first-order logic

Bridging the gap: F*

  • Functional programming language with effects

  • Semi-automated verification system using SMT

    • like Dafny, FramaC, Why3,
  • Interactive proof assistant based on dependent types

    • like Coq, Lean, Agda,

F* in action, at scale

  • Functional programming language with effects

    • F* is programmed in F*, but not (yet) verified
  • Semi-automated verification system

    • Project Everest: verify and deploy new, efficient HTTPS stack
      • miTLS*: Verified reference implementation of TLS (1.2 and 1.3)
      • HACL*: High-Assurance Cryptographic Library
      • Vale: Verified Assembly Language for Everest
  • Proof assistant based on dependent types

    • Fallback when SMT fails; also for mechanized metatheory:
      • MicroFStar: Fragment of F* formalized in F*
      • Wys*: Verified DSL for secure multi-party computations
      • ReVer: Verified compiler to reversible circuits

The current F* team

Microsoft Research, Inria Paris, MIT, Rosario, …

  • Danel Ahman
  • Benjamin Beurdouche
  • Karthikeyan Bhargavan
  • Tej Chajed
  • Antoine Delignat-Lavaud
  • Victor Dumitrescu
  • Cédric Fournet
  • Armaël Guéneau
  • Cătălin Hriţcu
  • Samin Ishtiaq
  • Markulf Kohlweiss
  • Qunyan Mangus
  • Kenji Maillard
  • Guido Martínez
  • Clément Pit-Claudel
  • Jonathan Protzenko
  • Tahina Ramananandro
  • Aseem Rastogi
  • Nikhil Swamy (benevolent dictator)
  • Christoph M. Wintersteiger
  • Santiago Zanella-Béguelin
  • Jean-Karim Zinzindohoué

The rest of this lecture

  • The functional core of F*

  • Verifying functional programs

  • Using very simple examples throughout

  • Small hands-on exercises here and there

The functional core of F*

  • Recursive functions

    val factorial : int -> int
    let rec factorial n = (if n = 0 then 1 else n * (factorial (n - 1)))
  • Inductive datatypes (immutable) and pattern matching

    type list (a:Type) =
      | Nil  : list a
      | Cons : hd:a -> tl:list a -> list a
    
    val map : ('a -> 'b) -> list 'a -> list 'b
    let rec map f x = match x with
      | [] -> []
      | h :: t -> f h :: map f t
  • Lambdas (unnamed, first-class functions)

    map (fun x -> x + 42) [1;2;3]

Refinement types

type nat = x:int{x>=0}
  • Refinements introduced by type annotations (code unchanged)

    val factorial : nat -> nat
    let rec factorial n = (if n = 0 then 1 else n * (factorial (n - 1)))
  • Logical obligations discharged by SMT (simplified)

    n >= 0, n <> 0 |= n - 1 >= 0
    n >= 0, n <> 0, (factorial (n - 1)) >= 0 |= n * (factorial (n - 1)) >= 0
  • Refinements eliminated by subtyping: nat<:int

    let i : int = factorial 42
    let f : x:nat{x>0}->int = factorial

Dependent types

  • Dependent function types ($\Pi$), here together with refinements:

    val incr : x:int -> y:int{x < y}
    let incr x = x + 1
  • Can express pre- and post- conditions of pure functions

    val incr : x:int -> y:int{y = x + 1}
  • Exercise: Can you find other types for incr?

Total functions in F*

  • The F* functions we saw so far were all total

  • Tot effect (default) = no side-effects, terminates on all inputs

    val factorial : nat -> Tot nat
    let rec factorial n = (if n = 0 then 1 else n * (factorial (n - 1)))
  • Quiz: How about giving this weak type to factorial?

    val factorial : int -> Tot int
  let rec factorial n = (if n = 0 then 1 else n * (factorial (n - 1)))
                                                              ^^^^^
  Subtyping check failed; expected type (x:int{(x << n)}); got type int

factorial (-1) loops! (int type in F* is unbounded)

Semantic termination checking

  • based on well-founded ordering on expressions (<<)
    • naturals related by < (negative integers unrelated)
    • inductives related by subterm ordering
    • lex tuples %[a;b;c] with lexicographic ordering
  • order constraints discharged by the SMT solver
  • arbitrary total expression as decreases metric
    val ackermann: m:nat -> n:nat -> Tot nat (decreases %[m;n])
    let rec ackermann n m =
      if m=0 then n + 1
      else if n = 0 then ackermann 1 (m - 1)
      else ackermann (ackermann (n - 1) m) (m - 1)
  • default metric is lex ordering of all (non-function) args
    val ackermann: m:nat -> n:nat -> Tot nat

The divergence effect (Dv)

  • We might not want to prove all code terminating

    val factorial : int -> Dv int
  • Some useful code really is not always terminating

    val eval : exp -> Dv exp
    let rec eval e =
      match e with
      | App (Lam x e1) e2 -> eval (subst x e2 e1)
      | App e1 e2 -> eval (App (eval e1) e2)
      | Lam x e1 -> Lam x (eval e1)
      | _ -> e
    let main = eval (App (Lam 0 (App (Var 0) (Var 0)))
                         (Lam 0 (App (Var 0) (Var 0))))
    ./Divergence.exe

F* effect system encapsulates effectful code

  • Pure code cannot call potentially divergent code

  • Only pure code can appear in specifications

    val factorial : int -> Dv int
        type tau = x:int{x = factorial (-1)}
                         ^^^^^^^^^^^^^^^^^^
    Expected a pure expression; got an expression ... with effect "DIV"
  • Sub-effecting: Tot t <: Dv t
    (e.g. divergent code can include pure code)

    incr 2 + factorial (-1) : Dv int

Verifying purely functional programs

Variant #1: intrinsically (at definition time)

  • Using refinement types (saw this already)
    val factorial : nat -> Tot nat
  • Can equivalently use pre- and post- conditions for this
    val factorial : x:int -> Pure int (requires (x >= 0))
                                    (ensures (fun y -> y >= 0))
  • Each computation type contains
    • effect (Pure, Div), result type (int), spec (e.g. pre and post)
  • Tot can be seen as just an abbreviation
    Tot t = Pure t (requires True) (ensures (fun _ -> True))

Verifying potentially divergent programs

The only variant: intrinsically (partial correctness)

  • Using refinement types
    val factorial : nat -> Dv nat
  • Or the Div computation type (pre- and post- conditions)
    val eval_closed : e:exp -> Div exp (requires (closed e))
                                     (ensures (fun e' -> Lam? e' /\ closed e'))
    let rec eval_closed e =
    match e with
    | App e1 e2 -> let Lam e1' = eval_closed e1 in
                   below_subst_beta 0 e1' e2;
                   eval_closed (subst (sub_beta e2) e1')
    | Lam e1 -> Lam e1
  • Dv just an abbreviation
    Dv t = Div t (requires True) (ensures (fun _ -> True))

Another way to look at this

  • Two classes of types

    • Value types (t): int, list int,
    • Computation types (C): Tot t   and   Dv t
  • Dependent function types of the form: x:t -> C

    • argument can't have side-effects, so value type
  • Two forms of refinement types

    • Refined value types: x:t{p}
    • Refined computation types:
      • Pure t pre post   and   Div t pre post
      • these will get more interesting for more interesting effects

Verifying purely functional programs

Variant #2: extrinsically using SMT-backed lemmas

let rec append (#a:Type) (xs : list a) (ys : list a) : Tot (list a) =
  match xs with
  | [] -> ys
  | x :: xs' -> x :: append xs' ys
let rec append_length (#a:Type) (xs : list a) (ys : list a) :
    Pure unit (requires True)
              (ensures (fun _ -> length (append xs ys) = length xs + length ys))
= match xs with
  | [] -> ()
  | x :: xs' -> append_length xs' ys
  • Syntax sugar (Lemma)
    let rec append_length (#a:Type) (xs : list a) (ys : list a) :
      Lemma (ensures (length (append xs ys) = length xs + length ys)) = ...

Often lemmas are unavoidable

let snoc l h = l @ [h]

val reverse: #a:Type -> list a -> Tot (list a)
let rec reverse (#a:Type) l =
  match l with
  | [] -> []
  | hd::tl -> snoc (reverse tl) hd
val rev_snoc: #a:Type -> l:list a -> h:a ->
  Lemma (reverse (snoc l h) == h::reverse l)
let rec rev_snoc (#a:Type) l h =
  match l with
  | [] -> ()
  | hd::tl -> rev_snoc tl h
val rev_involutive: #a:Type -> l:list a -> Lemma (reverse (reverse l) == l)
let rec rev_involutive (#a:Type) l =
  match l with
  | [] -> ()
  | hd::tl -> rev_involutive tl; rev_snoc (reverse tl) hd

Exercise: Summing: 0 + 1 + 2 + 3 + …

module Sum

open FStar.Mul

let rec sum_rec (n:nat) = if n > 0 then n + sum_rec (n-1) else 0

let sum_tot (n:nat) = ((n+1) * n) / 2

let rec sum_rec_correct (n:nat) : Lemma (sum_rec n = sum_tot n) =
  admit() (* replace this admit by a real proof *)
  if n > 0 then sum_rec_correct (n-1)

Verifying pure programs

Variant #3: extrinsically using proof terms

val preservation : #e:exp -> #e':exp -> #g:env -> #t:typ ->
       ht:(typing g e t) -> hs:step e e' -> Tot (typing g e' t) (decreases ht)
let rec preservation #e #e' #g #t (TyApp h1 h2) hs =
  match hs with
  | SBeta tx e1' e2' -> substitution_beta h2 (TyLam?.hbody h1)
  | SApp1 e2' hs1   -> TyApp (preservation h1 hs1) h2
  | SApp2 e1' hs2   -> TyApp h1 (preservation h2 hs2)
val progress : #e:exp -> #t:typ -> h:typing empty e t ->
                         Pure (cexists (fun e' -> step e e'))
                              (requires (~ (is_value e)))
                              (ensures (fun _ -> True)) (decreases h)
let rec progress #e #t h =
  match h with
  | TyApp #g #e1 #e2 #t11 #t12 h1 h2 ->
     match e1 with
     | ELam t e1' -> ExIntro (subst (sub_beta e2) e1') (SBeta t e1' e2)
     | _          -> let ExIntro e1' h1' = progress h1 in
                     ExIntro (EApp e1' e2) (SApp1 e2 h1')
  • Note: match exhaustiveness check also semantic (via SMT)

Functional core of F*

  • Variant of dependent type theory

    • $\lambda$, $\Pi$, inductives, matches, universe polymorphism
  • Recursion and semantic termination check

    • potential non-termination is an effect
  • Refinements

    • Refined value types: x:t{p}
    • Refined computation types: Pure t pre post
    • computationally and proof irrelevant, discharged by SMT
  • Subtyping and sub-effecting, Extensional equality

Abstract types: an interface

Stack.fsti

module Stack

  val stack : Type0  (* type stack *)

  val empty : stack
  val push : int -> stack -> stack
  val is_empty : stack -> bool
  val pop : stack -> option stack
  val top : stack -> option int

A client of this interface

StackClient.fst

module StackClient

  let main =
    let s0 = Stack.empty in
    let s1 = Stack.push 3 s0 in
    let s2 = Stack.push 4 s1 in
    Stack.top s2

An implementation using lists of ints

Stack.fst

module Stack

  let stack = list int
  let empty = []
  let push x xs = x :: xs
  let is_empty xs = match xs with
                    | [] -> true
                    | x::xs' -> false
  let pop xs = match xs with
               | [] -> None
               | x::xs' -> Some xs'
  let top xs = match xs with
               | [] -> None
               | x::xs' -> Some x

Clients cannot break abstraction

StackClientBad.fst

module StackClientBad

  let main =
    let s0 = Stack.empty in
    let s1 = Stack.push 3 s0 in
    2 :: s1
[hritcu@detained pure-fun]$ fstar.exe StackClientBad.fst
./StackClientBad.fst(6,9-6,11): (Error)
Expected expression of type "Prims.list ((*?u13*) _ s0 s1)";
got expression "s1" of type "Stack.stack"

Using refinement types for stacks

module RefinedStack

  abstract type stack = list int
  
  abstract val is_empty : stack -> Tot bool
  let is_empty = Nil?

  abstract val empty : s:stack{is_empty s}
  let empty = []
  
  abstract val push : int -> stack -> Tot (s:stack{~(is_empty s)})
  let push x xs = Cons x xs

  abstract val pop : s:stack{~(is_empty s)} -> Tot stack
  let pop = Cons?.tl
  
  abstract val top : s:stack{~(is_empty s)} -> Tot int
  let top = Cons?.hd
 

Client for RefinedStack

module RefinedStackClient

open RefinedStack

let main() : Tot stack =
  let s = push 1 (push 2 (push 3 empty)) in
  let t = top s in
  let s' = pop s in s'
  (* pop s' -- Subtyping check failed;
       expected type (s:stack{~(is_empty s)}); got type stack *)
  • Exercise: redesign RefinedStack interface so that this works

A more interesting variant (1)

module AbstractStack

  abstract type stack = list int
  abstract let is_empty : stack -> Tot bool = Nil?
  abstract let empty : s:stack{is_empty s} = []
  abstract let push (x:int) (xs:stack) : Tot (s:stack{~(is_empty s)}) = Cons x xs
  abstract let pop : s:stack{~(is_empty s)} -> Tot stack = Cons?.tl
  abstract let top : s:stack{~(is_empty s)} -> Tot int = Cons?.hd

  let top_push (i:int) (s:stack) :
    Lemma (top (push i s) = i) [SMTPat (top (push i s))] = ()

  let pop_push (i:int) (s:stack) :
    Lemma (pop (push i s) = s) [SMTPat (pop (push i s))] = ()

  let push_top_pop (s:stack{~(is_empty s)}) :
    Lemma (ensures (s = push (top s) (pop s))) = ()

A more interesting variant (2)

module AbstractStackClient

open AbstractStack

let main() : Tot stack =
  let s = push 1 (push 2 (push 3 empty)) in
  let t = top s in
  (* top_push 1 (push 2 (push 3 empty)); *)
  assert (t = 1);
  let s' = pop s in
  (* pop_push 1 (push 2 (push 3 empty)); *)
  pop s'

Next steps today

  • Cédric: Overview of Project Everest

  • Cătălin: A Gentle Introduction to F*:
    Verifying purely functional programs

  • Cătălin: Verifying Stateful Programs in F*

  • Cédric: Verifying crypto algorithms

  • Exercise session