We performed the theoretical study of circular dichroism (CD) in the second harmonic (SH) signal generated in nanostructures of different symmetries. In particular, we explored nonlinear response of dielectric AlGaAs nanoparticles and showed that even in the case of symmetric achiral shape circular dichroism is possible only for some specific shapes and crystalline lattice orientations. Using the apparatus of group theory, we compared dimer and C3v symmetric trimer structures, and explained the appearance of SH-CD in dimer, as well as the absence of dichroism in AlGaAs trimers. In summary, we proved, that knowing the general symmetry of the nanostructure with crystalline lattice is not enough, and detailed analysis of the eigenmodes and nonlinear polarization symmetry is required.