Professional Documents
Culture Documents
Albert Atserias
Universitat Politècnica de Catalunya
Barcelona, Spain
linear and semidefinite programming
approximation algorithms and computational complexity
logic and finite model theory
Part I
LINEAR PROGRAMMING
RELAXATIONS
Vertex cover
Problem:
Notation:
vc(G ).
Observe:
A ⊆ V is a vertex cover of G
iff
V \ A is an independent set of G
Linear programming relaxation
LP relaxation:
P
minimize u∈V xu
subject to
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V .
Notation:
fvc(G ).
Approximation
Approximation:
Integrality gap:
vc(G )
sup
G fvc(G )
Approximation
Approximation:
Integrality gap:
vc(G )
sup = 2.
G fvc(G )
Approximation
Approximation:
Integrality gap:
vc(G )
sup = 2.
G fvc(G )
Gap examples:
1. vc(K2n+1 ) = 2n,
2. fvc(K2n+1 ) = 12 (2n + 1).
LP tightenings
Add triangle inequalities:
P
minimize u∈V xu
subject to
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V ,
xu + xv + xw ≥ 2 for every triangle {u, v , w } in G .
LP tightenings
Add triangle inequalities:
P
minimize u∈V xu
subject to
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V ,
xu + xv + xw ≥ 2 for every triangle {u, v , w } in G .
Integrality gap:
Remains 2.
Gap examples:
Hierarchy:
Systematic ways of
generating all linear inequalities
that are valid over the integral hull.
Sherali-Adams and Lasserre/Sums-of-Squares Hierarchies
Hierarchy:
Systematic ways of
generating all linear inequalities
that are valid over the integral hull.
Given a polytope:
P = {x ∈ Rn : Ax ≥ b},
P Z = convexhull{x ∈ {0, 1}n : Ax ≥ b}.
Sherali-Adams and Lasserre/Sums-of-Squares Hierarchies
Hierarchy:
Systematic ways of
generating all linear inequalities
that are valid over the integral hull.
Given a polytope:
P = {x ∈ Rn : Ax ≥ b},
P Z = convexhull{x ∈ {0, 1}n : Ax ≥ b}.
P = P 1 ⊇ P 2 ⊇ · · · ⊇ P n−1 ⊇ P n = P Z
P k : Lasserre/Sums-of-squares (SOS) Hierarchy
P k : Lasserre/Sums-of-squares (SOS) Hierarchy
Given linear inequalities
L1 ≥ 0, . . . , Lm ≥ 0
P k : Lasserre/Sums-of-squares (SOS) Hierarchy
Given linear inequalities
L1 ≥ 0, . . . , Lm ≥ 0
produce all linear inequalities of the form
m
X n
X
Q0 + Lj Qj + (xi2 − xi )Qi = L ≥ 0
j=1 i=1
P k : Lasserre/Sums-of-squares (SOS) Hierarchy
Given linear inequalities
L1 ≥ 0, . . . , Lm ≥ 0
produce all linear inequalities of the form
m
X n
X
Q0 + Lj Qj + (xi2 − xi )Qi = L ≥ 0
j=1 i=1
where
X
2
Qj = Qj` with
`∈I
P k : Lasserre/Sums-of-squares (SOS) Hierarchy
Given linear inequalities
L1 ≥ 0, . . . , Lm ≥ 0
produce all linear inequalities of the form
m
X n
X
Q0 + Lj Qj + (xi2 − xi )Qi = L ≥ 0
j=1 i=1
where
X
2
Qj = Qj` with
`∈I
and
deg(Q0 ), deg(Lj Qj ), deg((xi2 − xi )Qi ) ≤ k.
P k : Lasserre/Sums-of-squares (SOS) Hierarchy
Given linear inequalities
L1 ≥ 0, . . . , Lm ≥ 0
produce all linear inequalities of the form
m
X n
X
Q0 + Lj Qj + (xi2 − xi )Qi = L ≥ 0
j=1 i=1
where
X
2
Qj = Qj` with
`∈I
and
deg(Q0 ), deg(Lj Qj ), deg((xi2 − xi )Qi ) ≤ k.
Then:
P k = {x ∈ Rn : L(x) ≥ 0 for each produced L ≥ 0}
P k : Sherali-Adams (SA) Hierarchy
Given linear inequalities
L1 ≥ 0, . . . , Lm ≥ 0
produce all linear inequalities of the form
m
X n
X
Q0 + Lj Qj + (xi2 − xi )Qi = L ≥ 0
j=1 i=1
where
X Y Y
Qi = c` xi (1 − xi ) with c` ≥ 0
`∈J i∈A` i∈B`
and
deg(Q0 ), deg(Lj Qj ), deg((xi2 − xi )Qi ) ≤ k.
Then:
P k = {x ∈ Rn : L(x) ≥ 0 for each produced L ≥ 0}
Example: triangles in P 3
For each triangle {u, v , w } in G :
Q0 +
(xu + xv − 1)Q1 +
(xu + xw − 1)Q2 +
(xv + xw − 1)Q3 +
(xu2 − xu )Q4 +
(xv2 − xv )Q5 +
(xw2 − xw )Q6
=?
(xu + xv + xw − 2).
Qi = ai + bi xu + ci xv + di xw + ei xu xv + fi xu xw + gi xv xw + hi xu xv xw
Solving P k
Lift-and-project:
k
• Step 1: lift from Rn up to R(n+1) and linearize the problem
k
• Step 2: project from R(n+1) down to Rn
Proposition:
Proof:
1. for SA-P k : by linear programming
2. for SOS-P k : by semidefinite programming
An Important Open Problem
Define
Define
Open problem:
vc(G ) ?
sup 4
<2
G sos fvc(G )
What’s Known
What’s Known
Known (conditional hardness):
• 1.0001-approximating vc(G ) is NP-hard by PCP Theorem
What’s Known
Known (conditional hardness):
• 1.0001-approximating vc(G ) is NP-hard by PCP Theorem
• 1.36-approximating vc(G ) is NP-hard
What’s Known
Known (conditional hardness):
• 1.0001-approximating vc(G ) is NP-hard by PCP Theorem
• 1.36-approximating vc(G ) is NP-hard
• 2-approximating vc(G ) is NP-hard assuming UGC
What’s Known
Known (conditional hardness):
• 1.0001-approximating vc(G ) is NP-hard by PCP Theorem
• 1.36-approximating vc(G ) is NP-hard
• 2-approximating vc(G ) is NP-hard assuming UGC
Gap examples:
Gap examples:
COUNTING LOGIC
Bounded-Variable Logics
P1 (x, y ) := E (x, y )
P2 (x, y ) := ∃z1 (E (x, z1 ) ∧ P1 (z1 , y ))
P3 (x, y ) := ∃z2 (E (x, z2 ) ∧ P2 (z2 , y ))
..
.
Pi+1 (x, y ) := ∃zi (E (x, zi ) ∧ Pi (zi , y ))
..
.
Example
Paths:
P1 (x, y ) := E (x, y )
P2 (x, y ) := ∃z1 (E (x, z1 ) ∧ P1 (z1 , y ))
P3 (x, y ) := ∃z2 (E (x, z2 ) ∧ P2 (z2 , y ))
..
.
Pi+1 (x, y ) := ∃zi (E (x, zi ) ∧ Pi (zi , y ))
..
.
Counting witnesses:
Counting witnesses:
C k -equivalence:
Color-refinement:
Color-refinement:
Notation:
Color-refinement:
Notation:
1. G ∼
= H,
2. there exists permutation matrix P such that P T GP = H,
3. there exists permutation matrix P such that GP = PH.
LP characterization of color-refinement
Isomorphisms:
1. G ∼
= H,
2. there exists permutation matrix P such that P T GP = H,
3. there exists permutation matrix P such that GP = PH.
LP relaxation of ∼
=:
iso(G , H) : GS = SH
Se = e T S = e
S ≥ 0.
LP characterization of color-refinement
Isomorphisms:
1. G ∼
= H,
2. there exists permutation matrix P such that P T GP = H,
3. there exists permutation matrix P such that GP = PH.
LP relaxation of ∼
=:
iso(G , H) : GS = SH
Se = e T S = e
S ≥ 0.
Theorem [Tinhofer]
G ≡R H if and only if G ≡F H.
Higher levels of SA Hierarchy
G ≡SA
k H : the degree-k SA level of iso(G , H) is feasible.
Higher levels of SA Hierarchy
G ≡SA
k H : the degree-k SA level of iso(G , H) is feasible.
G ≡SA C SA
k H =⇒ G ≡k H =⇒ G ≡k−1 H.
Higher levels of SA Hierarchy
G ≡SA
k H : the degree-k SA level of iso(G , H) is feasible.
G ≡SA C SA
k H =⇒ G ≡k H =⇒ G ≡k−1 H.
Moreover:
Theorem [AA and Ochremiak 2018]: There exists c > 1 such that:
APPLICATIONS
Local LPs (and SDPs)
k-local LP:
P
u∈V xu ≤ W
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V .
Example 1: fractional vertex cover
P
u∈V xu ≤ W
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V .
More examples:
1. maximum flows (2-local)
2. if P is r -local LP, then sak -P is rk-local LP.
3. if P is r -local LP, then sosk -P is rk-local SDP.
Back to integrality gaps for vertex cover
Goal:
For large k and every > 0 find graphs G and H such that
1. G ≡Cc·2k H
2. vc(G ) ≥ (2 − )vc(H)
Back to integrality gaps for vertex cover
Goal:
For large k and every > 0 find graphs G and H such that
1. G ≡Cc·2k H
2. vc(G ) ≥ (2 − )vc(H)
vc(G )
sup =2
G sosk fvc(G )
Back to integrality gaps for vertex cover
Goal:
For large k and every > 0 find graphs G and H such that
1. G ≡Cc·2k H
2. vc(G ) ≥ (2 − )vc(H)
vc(G )
sup =2
G sosk fvc(G )
Proof:
vc(G ) ≥ (2 − )vc(H) by 2.
≥ (2 − )sosk fvc(H) obvious
≥ (2 − )sosk fvc(G ) by 1. and 2-locality
GOAL
For large k and every > 0 find graphs G and H such that
1. G ≡Cc·2k H
2. vc(G ) ≥ (2 − )vc(H)
A weak (easy) case: k = 1 with gap = 2
Choose:
G = any d-regular expander graph (i.e., λ2 (G ) λ1 (G )),
H = any d-regular bipartite graph.
A weak (easy) case: k = 1 with gap = 2
Choose:
G = any d-regular expander graph (i.e., λ2 (G ) λ1 (G )),
H = any d-regular bipartite graph.
Then:
vc(G ) = (1 − )n by expansion
vc(H) = n/2 by bipartition
G ≡R H by regularity
G ≡C2 H by Tinhofer’s Theorem
A weak (easy) case: k = 1 with gap = 2
Choose:
G = any d-regular expander graph (i.e., λ2 (G ) λ1 (G )),
H = any d-regular bipartite graph.
Then:
vc(G ) = (1 − )n by expansion
vc(H) = n/2 by bipartition
G ≡R H by regularity
G ≡C2 H by Tinhofer’s Theorem
PROOF INGREDIENTS
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
• A ∈ Fm×n
2 and b ∈ Fn2
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
• A ∈ Fm×n
2 and b ∈ Fn2
• every row of A has at most three 1’s
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
• A ∈ Fm×n
2 and b ∈ Fn2
• every row of A has at most three 1’s
• every subset of m equations has at least δn unique variables
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
• A ∈ Fm×n
2 and b ∈ Fn2
• every row of A has at most three 1’s
• every subset of m equations has at least δn unique variables
• every candidate solution satisfies at most 21 + equations
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
• A ∈ Fm×n
2 and b ∈ Fn2
• every row of A has at most three 1’s
• every subset of m equations has at least δn unique variables
• every candidate solution satisfies at most 21 + equations
Probabilistic construction:
1. set m = cn for a large constant c = c()
2. choose three ones uniformly at random in each row of A
3. choose b uniformly at random in Fn2 .
1/3: Locally consistent systems of linear equations
Ingredient 1: A linear system Ax = b over F2 where:
• A ∈ Fm×n
2 and b ∈ Fn2
• every row of A has at most three 1’s
• every subset of m equations has at least δn unique variables
• every candidate solution satisfies at most 21 + equations
Probabilistic construction:
1. set m = cn for a large constant c = c()
2. choose three ones uniformly at random in each row of A
3. choose b uniformly at random in Fn2 .
Half-deterministic construction:
1. set m = cn for a large constrant c = c()
2. let A be incidence matrix of bipartite expander
3. choose b uniformly at random in Fn2 .
2/3: Indistinguishable systems of linear equations
Construction of S0 :
1. start with Ax = b from previous section
2. duplicate each variable x 7→ (x (0) , x (1) )
3. replace each equation xi + xj + xk = b by 8 equations
(u) (v ) (w )
xi + xj + xk =b+u+v +w
2/3: Indistinguishable systems of linear equations
Construction of S0 :
1. start with Ax = b from previous section
2. duplicate each variable x 7→ (x (0) , x (1) )
3. replace each equation xi + xj + xk = b by 8 equations
(u) (v ) (w )
xi + xj + xk =b+u+v +w
Construction of S1 :
1. same but start with Ax = 0 (the homogeneous system)
3/3: Reduction to vertex cover
Construction:
vc(G )
sup > 1.36?
G sos4 fvc(G )
Open Problem 2