The authors obtain the Fekete-Szegő inequality (according to parameters $s$ and $t$ in the region ${s}^{2}+st+{t}^{2}<3$, $s\ne t$ and $s+t\ne 2$, or in the region ${s}^{2}+st+{t}^{2}>3,$
$s\ne t$ and $s+t\ne 2$) for certain normalized analytic functions $f\left(z\right)$ belonging to $k{\text{-UST}}_{\lambda ,\mu}^{n}(s,t,\gamma )$ which satisfy the condition $$\Re \left\{\frac{(s-t)z{\left({D}_{\lambda ,\mu}^{n}f\left(z\right)\right)}^{\text{'}}}{{D}_{\lambda ,\mu}^{n}f\left(sz\right)-{D}_{\lambda ,\mu}^{n}f\left(tz\right)}\right\}>k\left|\frac{(s-t)z{\left({D}_{\lambda ,\mu}^{n}f\left(z\right)\right)}^{\text{'}}}{{D}_{\lambda ,\mu}^{n}f\left(sz\right)-{D}_{\lambda ,\mu}^{n}f\left(tz\right)}-1\right|+\gamma ,\phantom{\rule{1.0em}{0ex}}z\in \mathcal{U}.$$
Also certain applications of the main result a class of functions defined by the Hadamard product (or convolution) are given. As a special case of this result, the Fekete-Szegő inequality for a class of functions defined through fractional derivatives is obtained....