The modified zeta functions $\sum_{n \in K} n^{-s}$, where $K \subset \N$,
converge absolutely for $\Re s > 1/2$. These generalise the Riemann zeta
function which is known to have a meromorphic continuation to all of $\C$ with
a single pole at $s=1$. Our main result is a characterisation of the modified
zeta functions that have pole-like behaviour at this point. This behaviour is
defined by considering the modified zeta functions as kernels of certain
integral operators on the spaces $L^2(I)$ for symmetric and bounded intervals
$I \subset \R$. We also consider the special case when the set $K \subset \N$
is assumed to have arithmetic structure. In particular, we look at local $L^p$
integrability properties of the modified zeta functions on the abscissa $\Re
s=1$ for $p \in [1,\infty]$.