Let $p\geqslant 2$ be a prime and $\mathbb{Z}_p$ be the ring of $p$-adic integers. For any $\alpha,\beta,z\in \mathbb{Z}_p$, define $f_{\alpha,\beta}(z)=\alpha z+\beta$. The first part of this paper studies all minimal subsystems of semigroup dynamical systems $(\mathbb{Z}_p,G)$ when $f_{\alpha_1,\beta_1}$ and $f_{\alpha_2,\beta_2}$ are commutative, where the semigroup $G=\{f_{\alpha_1,\beta_1}^n \circ f_{\alpha_2,\beta_2}^m: m,n \in \mathbb{N}\}$. In particular, we find the semigroup dynamical system $(\mathbb{Z}_p,G)\ (p\geqslant 3)$ is minimal if and only if $(\mathbb{Z}_p,f_{\alpha_1,\beta_1})$ or $(\mathbb{Z}_p,f_{\alpha_2,\beta_2})$ is minimal and we determine all the cases that $(\mathbb{Z}_2,G)$ is minimal. In the second part, we study weakly essentially minimal affine semigroup dynamical systems on $\mathbb{Z}_p$, which is a kind of minimal semigroup systems without any minimal single action. It is shown that such semigroup is non-commutative when $p\geqslant 3$. Moreover, for a fixed prime $p$, we find the least number of generators of a weakly essentially minimal affine semigroup on $\mathbb{Z}_p$. We show that such number is $2$ for $p=2$ and $3$ for $p=3$. Also, we show that such number is not greater than $p$.