Mapping the neuronal wiring diagrams in the brain at multiple spatial scales has been one of the major brain mapping objectives. Macro-scale medical imaging modalities such as diffusion tensor imaging (DTI) and meso-scale biological imaging such as serial two-photon tomography have emerged as the prominent tools to reveal structural connectivity patterns at multiple scales. However, a significant gap that whether/how DTI data and microscopic data are correlated with each other for the s ame species of mammalian brains,e.g., mouse brains, has been rarely explored. To bridge this knowledge gap, this work aims to construct multi-modal mouse brain connectomes via joint modeling of macro-scale DTI data and meso-scale neuronal tracing data. Specifically, the high-resolution DTI data and its streamline tractography result are mapped to the Allen Mouse Brain Atlas, in which the high-density axonal projections were already mapped by microscopic serial two-photon tomography. Then, multi-modal connectomes were constructed and the multi-view spectral clustering method is employed to assess consistent and discrepant connectivity patterns across the multi-scale multi-modal connectomes. Experimental results demonstrated the importance of fusing multimodal, multi-scale imaging modalities for structural connectivity and connectome mapping.