Let $R \in M_n({\Bbb Z})$ be an expanding matrix and ${\cal D}=\{0, a_1, a_2, \cdots, a_{N-1}\}u \equiv \{0, 1, \cdots, N-1\}u \ ({\rm mod}N) $ be a $N$-element digit set, where $u\in {\Bbb Z}^n\setminus\{0\}$. In this paper, we study the spectral property of the self-affine measures $\mu_{R, {\cal D}}$ which is generated by ${\cal D}$ and $R$, and obtain a sufficient condition such that $\mu_{R, {\cal D}}$ is a spectral measure. Moreover, for a special case, we give a necessary and sufficient condition such that $\mu_{R, {\cal D}}$ is a spectral measure, and the exact spectrum of $\mu_{R, {\cal D}}$ is given.