Gumbel 重参数化相关性质证明

2023-12-19 00:19:17

Gumbel 的采样过程:

z = a r g m a x i { g i + l o g ( π i ) } , g i = ? l o g ( ? l o g ( u i ) ) , u i ~ U ( 0 , 1 ) z=argmax_i \{g_i + log(\pi_i)\}, g_i = -log(-log(u_i)),u_i\sim U(0, 1) z=argmaxi?{gi?+log(πi?)},gi?=?log(?log(ui?)),ui?U(0,1)

采样得到的随机变量满足一下分布:

g i ~ G u m b l e ( 0 , 1 ) ( 1 ) g_i \sim Gumble(0, 1) \quad (1) gi?Gumble(0,1)(1)

h i = g i + l o g ( π i ) ~ G u m b l e ( l o g ( π i ) , 1 ) ( 2 ) h_i = g_i + log(\pi_i)\sim Gumble(log(\pi_i), 1) \quad (2) hi?=gi?+log(πi?)Gumble(log(πi?),1)(2)

证明过程:

P ( u ) = P ( U ≤ u ) = u , u ∈ ( 0 , 1 ) P(u) =P(U\le u)= u, u\in(0, 1) P(u)=P(Uu)=u,u(0,1)

G = ? l o g ( ? l o g ( U ) ) , u ∈ ( 0 , 1 ) G = -log(-log(U)), u\in(0, 1) G=?log(?log(U)),u(0,1)

P ( g ) = P ( G ≤ g ) = P ( ? l o g ( ? l o g ( U ) ) ≤ g ) P(g) =P(G\le g) = P(-log(-log(U))\le g) P(g)=P(Gg)=P(?log(?log(U))g)

= P ( U ≤ e x p ( ? e x p ( ? g ) ) ) =P(U\le exp(-exp(-g))) =P(Uexp(?exp(?g)))

= e x p ( ? e x p ( ? g ) ) = exp(-exp(-g)) =exp(?exp(?g))

P ( g ) = e x p ( ? e x p ( ? g ) ) P(g) = exp(-exp(-g)) P(g)=exp(?exp(?g))

g i ~ G u m b l e ( 0 , 1 ) g_i\sim Gumble(0, 1) gi?Gumble(0,1)

h i = g i + l o g ( π i ) ~ G u m b l e ( l o g ( π i ) , 1 ) h_i = g_i + log(\pi_i)\sim Gumble(log(\pi_i), 1) hi?=gi?+log(πi?)Gumble(log(πi?),1)

P ( Z = z ) = π i ( 3 ) P(Z=z) = \pi_i \quad(3) P(Z=z)=πi?(3)

证明过程:

P ( Z = z ∣ U z = u z ) = ∏ i ≠ z P ( H i < g z + l o g ( π z ) ) P(Z=z | U_z = u_z) = \prod_{i\ne z} P(H_i < g_z + log(\pi_z)) P(Z=zUz?=uz?)=i=z?P(Hi?<gz?+log(πz?))

= ∏ i ≠ z P ( G i + l o g ( π i ) < g z + l o g ( π z ) ) =\prod_{i\ne z} P(G_i + log(\pi_i) < g_z + log(\pi_z)) =i=z?P(Gi?+log(πi?)<gz?+log(πz?))

= ∏ i ≠ z P ( U i < u z p i / p z ) = \prod_{i\ne z} P(U_i < u_z^{p_i/p_z}) =i=z?P(Ui?<uzpi?/pz??)

= ∏ i ≠ z u z p i / p z = u z 1 / p z ? 1 = \prod_{i\ne z} u_z^{p_i/p_z} = u_z^{1/p_z - 1} =i=z?uzpi?/pz??=uz1/pz??1?

P ( Z = z ) = ∫ 0 1 P ( Z = z ∣ U z = u z ) P ( U z = u z ) d u z P(Z = z) = \int_0^1 P(Z=z|U_z = u_z)P(U_z=u_z) du_z P(Z=z)=01?P(Z=zUz?=uz?)P(Uz?=uz?)duz?

= ∫ 0 1 u z 1 / p z ? 1 ? 1 ? d u z = \int_0^1 u_z^{1/p_z - 1} * 1 * du_z =01?uz1/pz??1??1?duz?

= 1 1 / p z u z 1 / p z ∣ 0 1 = \frac{1}{1/p_z}u_z^{1/p_z}|_0^1 =1/pz?1?uz1/pz??01?

= p z = p_z =pz?

文章来源:https://blog.csdn.net/gaofeipaopaotang/article/details/135072123
本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。