Sunday, August 5, 2018

[Code snippet]: Planet API Image Download Url

Be The First To Comment

from requests.auth import HTTPBasicAuth
import os
import requests


item_id = "20161109_173041_0e0e"
item_type = "PSScene3Band"


os.environ['PLANET_API_KEY'] = '3b711bededf6485a0' #not a real id
asset_type = "visual"
item_url = 'https://api.planet.com/data/v1/item-types/{}/items/{}/assets'.format(item_type, item_id)

# Request a new download URL
result = requests.get(item_url, auth=HTTPBasicAuth(os.environ['PLANET_API_KEY'], ''))
download_url = result.json()[asset_type]['location']

print(download_url)

Friday, May 25, 2018

[Code Snippet] Find Custom Role of users that are in ArcGIS portal - ArcGIS

Be The First To Comment
from arcgis.gis import *

self.portalInfo = GIS(self.portalUrl, self.userName, self.password)

self.portalUsers = self.portalInfo.users.search('')

users = self.portalUsers

roleManager = arcgis.gis.RoleManager(self.portalInfo)

roles = roleManager.all()

for user in users:

if hasattr(user,'roleId'):

for role in roles:

if(user.roleId == role.role_id):

print(user.username,user.role,role.name)

Wednesday, April 11, 2018

Code snippet: Get DateTime from UTC timestamp in ArcGIS Online

Be The First To Comment
 ArcGIS Online stores all date values in UTC.  A short snippet to convert UTC timestamp to DateTime using Python.

      from datetime import datetime

      utcTimeStamp = 1472218452855

      date = datetime.fromtimestamp(utcTimeStamp / 1e3)
      
      print(date)

output: 2016-08-26 09:34:12.855000

Friday, February 2, 2018

[Code snippet] Extract Feature Layer Fields from Map Document (Mxd) using ArcPy

Be The First To Comment
This code snippet is for getting a list of fields in a feature layer from a map document in ArcPy 10.4.


1:  #Import arcpy mapping library  
2:  import arcpy.mapping;  
3:    
4:  # Map doc path  
5:  mxdPath =r"C:\MyMxd.mxd"  
6:    
7:  #Open map document  
8:  document = arcpy.mapping.MapDocument(mxdPath)  
9:    
10:  #Extract all dataframes inside a map document  
11:  dataFrameList = arcpy.mapping.ListDataFrames(document)  
12:    
13:  #Loop through all DF   
14:  for dataFrame in dataFrameList:  
15:  #Extact all layers in a dataframe   
16:    layerList = arcpy.mapping.ListLayers(document, None, dataFrame)  
17:      
18:    #Loop through all Layers     
19:    for layer in layerList:  
20:      print (dataFrame.name+"--"+layer.name +"--"+layer.dataSource)  
21:        
22:      #Extract all fields in a layer   
23:      fieldList = arcpy.ListFields(layer.dataSource, None, None)  
24:        
25:      #Loop through and print field properties       
26:      for field in fieldList:  
27:         print(field.name,field.aliasName,field.type,field.length,field.required,field.precision)  

Monday, October 23, 2017

Upgrade or Install Python PIP in ArcGIS Desktop 10.4.1

Be The First To Comment
ArcGIS  Desktop 10.4.1’s default pip, C:\Program Files (x86)\Python27\ArcGIS10.4\Scripts  failed to install the packages throwing following error –

Fatal error in launcher: Unable to create process using '"C:\Python27\ArcGIS10.4
\python.exe" "C:\Python27\ArcGISx6410.4\Scripts\pip.exe" 
The following steps to make the pip to work

Step 1. Open command terminal as Administrator and see if pip works
      Make sure 
            C:\Program Files (x86)\Python27\ArcGIS10.4\Lib;
            C:\Program Files (x86)\Python27\ArcGIS10.4\Scripts;

            C:\Program Files (x86)\Python27\ArcGIS10.4
      are in path            
  
           pip install <package_name> , no luck [ You can Jump directly to #4]
        
Step 2. Check the pip version
            pip --version
            (pip9.0.1 as of today)

Step 3. Try to upgrade pip

            pip install --upgrade pip

            The command run successfully, but did not upgrade pip. Every time terminal complains –
You are using pip version 7.0.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Step 4. Installing pip from beginning although it ships automatically in Python 2.7 ( Arcgis 10.)
To install pip, securely download get-pip.py.
Then run the following:

python get-pip.py
or
python get-pip.py --trusted-host pypi.python.org ( if SSL certification error occurs)

Step 5. Check pip version

pip --version

pip 9.0.1 from C:\Python27\ArcGIS10.4\lib\site-packages (python 2.7)
( You should have latest version of pip)

Step 6. Now use pip to install packages

pip install <package_name> --trusted-host pypi.python.org

           

Tuesday, May 2, 2017

Python code snippet : Merge multiple CSVs (Column Wise) into a large CSV file using ID

Be The First To Comment
Following is the python code snippet to merge multiple CSVs into a large CSV (column wise) using CSV's unique ID field. All input CSV's must have same length.

 import csv
 import itertools
 
 outCombinedStatTxtName ="FormattedTxtResults.csv"  
 tempCsvFiles = glob.glob(str(TempFolderPath)+'\*.csv')  
 openedFileList = [open(fn, 'rb') for fn in tempCsvFiles]  
 readers = [csv.reader(fn) for fn in openedFileList]  
   
 result = open(outCombinedStatTxtName, 'wb')  
 writer = csv.writer(result,delimiter=',')  
   
 multipleIndices = False  
 for row_chunks in itertools.izip(*readers):  
   tempFormattedRow = list(itertools.chain.from_iterable(row_chunks))  
   fidIndices = [i for i, x in enumerate(tempFormattedRow) if x == "ID"]  
   if(len(fidIndices) > 1):  
     fidIndices.pop(0)  
     redundantIndices = tuple(fidIndices)  
     multipleIndices = True  
       
   if(multipleIndices):  
     tempFormattedRow = [ tempFormattedRow[i] for i in xrange(len(tempFormattedRow)) if i not in set(redundantIndices) ]      
     writer.writerow(tempFormattedRow)  
   else:  
     writer.writerow(tempFormattedRow)  
 result.close()     
 [fn.close() for fn in openedFileList]  


Friday, May 15, 2015

Tiler tools, an alternative scripts to Gdal2tiles.py for creating raster tiles from digital maps

Be The First To Comment
Other day I was working to generate the Google Map based raster tiles using Gdal2tiles for a large raster file, 30GB in size. Gdal2tiels complains about the following error, which I had no clue.

Then fiddling with others tilers available I played with the Tilers-tools, python Scripts for raster tile sets from digital maps with GDAL dependency. After spending few days hit and trial, finally I am able to generate raster tiles compatible with Google Maps. Here I am sharing the steps that I follow to produce tiles using the Tiler tool. Hope it will help some of the OSGeo users. I have divided tile generation steps into two groups, dependency installation and data preparation with a presumption that you are already familiar with GDAL.

A] DEPENDENCY INSTALLATION

STEP 1. Install Python geospatial packages

Install the dependencies as described-

For my project, I used 32 bit installers and Python 2.7. Strictly follow the python Geospatial package dependency installation orders and do not mix the 64 bit and 32 bit dependencies

STEP 2. Install the Tiler tools

Download and unzip the Tiler tool from: http://sourceforge.net/p/tilers-tools/wiki/WinInstall/

STEP 3. Set Python and Gdal path in Tiler’s bat file

Open : tilers-tools.bat from unzipped Tiler tool
         set PYTHON=C:\Python27 ( As you assigned the path in Step 1)
         set GDAL=C:\Program Files (x86)\GDAL ( As you assigned the path in Step 1)

STEP 4. Environmental variable setting on your machine

Set GDAL_DATA & path in your environmental variable setting.

                   Variable          Value

       GDAL_DATA = C:\Program Files (x86)\GDAL\gdal-data

        path = C:\Program Files (x86)\GDAL

Tuesday, May 12, 2015

Easy 5 steps to get thumbnails out of Pdf using Python

Be The First To Comment
1) Install Ghost script before python dependencies installation from: http://www.a-pdf.com/convert-to-pdf/gs.exe

2) Make yourself familier with Wand, a ctypes-based simple ImageMagick binding for Python.

3) Install Wand - pip install wand

4) Use the following script..
 #Install Ghost script before python package's installation   
 #from: http://www.a-pdf.com/convert-to-pdf/gs.exe  

 from wand.image import Image  
 from wand.display import display  
 fileDirectory = "D:/files/"  
 inFileName="myInputfile.pdf"  
 outFileName="myOutputfile.png"  
 imageFromPdf = Image(filename=fileDirectory+inFileName)  
 pages = len(imageFromPdf.sequence)  
 print(pages)  
 image = Image(  
   width=imageFromPdf.width,  
   height=imageFromPdf.height * pages  
 )  
 for i in range(pages):  
   image.composite(  
     imageFromPdf.sequence[i],  
     top=imageFromPdf.height * i,  
     left=0  
   )  
 image.format="png"  
 image.save(filename=fileDirectory+outFileName)  
 display(image)  



5) Let me know if you have any trouble.

Tuesday, February 19, 2013

Python Scripting for ArcGIS

Be The First To Comment

Python Scripting for ArcGIS is a new text from Esri Press by Paul A. Zandbergen (2013). It isn’t the first Python book for the geospatial community or even focused on ArcGIS, but it is the first that has the Esri logo on it. Much like other recent books on Geo/Python we have seen, it focuses on integrating an introduction to Python with the industry specific materials. As Frank mentioned when he highlighted the book in a previous podcast, this allows users to gain exposure to Python, but it doesn’t fall back on the (in my opinion) bad habit of most programming texts of spending half of the book on the language and concepts before even getting to the application in the specific area. There is a time and place for that approach in Python specific books. When you add another software library to a book, then use it from the get go.

Thursday, January 10, 2013

pyKML - a Python library for generating and parsing KML

Be The First To Comment


pyKML is an open source Python library for generating, parsing, and modifying KML, the geo-spatial data language used by Google Earth, Google Maps and a number of other GIS platforms.

pyKML helps working with large and complex KML documents by leveraging the use of basic programming constructs (looping, branching, etc.). In this regard pyKML is similar to libkml, Google’s open source C++ library, but takes advantage of the highly readable syntax of the Python programming language and the processing capabilities of the popular lxml Python library.


pyKML v0.1.0 documentation :  http://packages.python.org/pykml/



Wednesday, January 9, 2013

GIS Programming for GIS Analyst using Python

Be The First To Comment


Penn state has a nice collection of ArcGIS & Python programming step by step guide for their GEOG 485 students, which is also available for non-students via Penn state website.  This course’s is main aim is to make student able to use the Python scripting language to automate GIS tasks in ArcMap, assemble ArcView geoprocessing tools into models to solve GIS problems, and run the tools from scripts to automate GIS tasks. 

Although no previous programming experience is assumed, the course's target is to make a student as a GIS programmer/analyst in ArcGIS and Python environment. 

The course is basically divided into four parts:

Wednesday, May 25, 2011

Raster Misalignment with Base Data in ArcMap10

Be The First To Comment
In the early Friday morning of mid May, I got an email from one of my team member about raster misalignment problem in ArcGIS10. I also tried to overlay couples of previously working Tiff and Grid raster files in ArcMap9.3 and ArcMap10. The ArcMap9.3 overlay raster files perfectly aligned as we all desired, but ArcMap10 did not.
Unaligned

Aligned

From the ESRI website, I got to know that the issue of misalignment of Tiff in ArcMap10 is a bug in ArcGIS 10. The ESRI team announced two solutions to solve Tiff shift into wrong geographic locations:

Tuesday, June 1, 2010

Getting MODIS Image Automatically From FTP in Python

7 Comments
# This is a Python script that automatically downloads historical
# MODIS data from the LP DAAC FTP site
# This version should work for all of the tiled datasets
# It is currently hard-coded to downloaded specific MODIS tiles for
# the northern Great Plains & upper midwest

# Initailly historical date for Data transfer must be set on "lpdacc.txt".



import os, ftplib,sys,string


# Login information for accessing the LP DAAC FTP site
Hostname = "e4ftl01u.ecs.nasa.gov"
Username = "anonymous"
Password = "@anonymous"

# Get user inputs
# Base directory for the MODIS data

# Basedir = input("Enter the LP DAAC directory containing the dataset you want to download:")
Basedir="MOLT/MOD11A2.005"
print "The LP DAAC directory containing the dataset you want to download" +str(Basedir)


# Local directory for data storage
#Hdfdir = input("Enter the local directory where you want to store the hdf files:")
Hdfdir=r"H:\MODIS_LST_NDVI\MOD11A2\\"
print "The local directory where you want to store the hdf files" +str(Hdfdir)


# Empty lists for the collector functions
Dirlist = []
Filelist = []

mylog=open(r"H:\MODIS_LST_NDVI\MOD11A2\mylog.txt",'w',)

# Assigning Global ariables
i=0
k=0
flag=0

try:

# Define helper functions that are used to read files/subdirectories from the
# ftp site and store them as lists
def collector(line = ''):
global Dirlist
Dirlist.append(line)

def collector2(line = ''):
global Filelist
Filelist.append(line)



# Open ftp connection
ftp = ftplib.FTP(Hostname,Username,Password)

# Go to the directory containing the dataset of interest
ftp.cwd(Basedir)

# Involke the LIST ftp function, calling the collector function to store the
# results to Dirlist in list format
ftp.retrlines("LIST", collector)

# Get Directory listing only (Without including the sub directories)
mainDirlist=[]
myDirlist=[]
ftp.dir(mainDirlist.append)
myDirlist=mainDirlist[1:]
# parsing the Directory list
dirInfo=""


for mainDir in myDirlist:
#parsing the directory name only[2002.12.03]
mainDirname= mainDir[37:47]
dd=mainDirname[8:10]
mm=mainDirname[5:7]
yyyy=mainDirname[0:4]
#Extracging yyyy mm dd to compare with log file information
dirInfo=str(yyyy)+str(mm)+str(dd)
print mainDirname
print dirInfo

# Read the log file to retrive the information of latest downloaded data
logFileread=open(r"H:\MODIS_LST_NDVI\MOD11A2\lpdaac.txt",'r')
logFileread.seek(0)
logInfo=int(logFileread.read())
logFileread.close()

print"Local Drive Recent Log Dir # "+str(logInfo)
mylog.write("Local Drive Recent Log Dir # "+str(logInfo)+'\n')

# Coparing the logfile(already downloaded data) with recent datas in the ftp
if(int(logInfo) print "current path -->"+str(ftp.pwd())
if(flag==1): # if flag matches the criteria reset the counters
ftp.cwd("..")
k=0
flag=0
Filelist = []

# List all the files in the subdirectory
path=str(mainDirname)
ftp.cwd(path)
print "New path -->"+str(ftp.pwd())
# Filelist = ftp.dir()
##FTP Directory bhitra chire pni file ma chire ko chhina
ftp.retrlines("LIST", collector2)
#ftp.retrlines("LIST")
# Download data from the MODIS tiles that we are interested in
for Currow2 in Filelist:
Splitrow2 = Currow2.split()
Permissions = Splitrow2[0]

# Skip over the jpeg browse images - some of these cause problems
if Permissions[0:3] == "-rw":
Directories = Splitrow2[1]
Group = Splitrow2[2]
Size = Splitrow2[3]
Month = Splitrow2[4]
Date = Splitrow2[5]
Time = Splitrow2[6]
Filename = Splitrow2[7]
mylog.write(Filename)
LocalFile = Hdfdir + Filename
Splitfname = Filename.split(".")
# Split the header file name into its various components
Splitfname = Filename.split(".")
Mdataset = Splitfname[0]
Maqdate = Splitfname[1]
Mlocation = Splitfname[2]
Mprocdate = Splitfname[3]
Mext1 = Splitfname[4]
Mext2 = Splitfname[5]


# Pull out the horizontal and vertical tile numbers
Htile = Mlocation[1:3]
Vtile = Mlocation[4:6]


# Only retrieve data for the three tiles covering the NGP/Upper Midwest
if (((Htile == "09") & (Vtile == "04"))|((Htile == "10") & (Vtile == "04"))|((Htile == "11") & (Vtile == "04"))|((Htile == "12") & (Vtile == "04"))|((Htile == "09") & (Vtile == "05"))|((Htile == "10") & (Vtile == "05"))|((Htile == "11") & (Vtile == "05"))):
# Retrieve the hdf and xml files and place them in the local directory

ftp.retrbinary("RETR " + Filename, open(LocalFile, "wb").write)

# Write download information in the log file
logFwrite=open(r"H:\MODIS_LST_NDVI\MOD11A2\lpdaac.txt",'w')
logFwrite.seek(0)
logFwrite.write(dirInfo)
logFwrite.close()

k=k+1
if(k==14): # value of k should be no of tiles*2[for *.hdf and *.hdf.xml ]
flag=1

else:
print i
i=i+1
print "loginfor-->"+str(logInfo)
print "dirInfor-->"+str(dirInfo)
print "Dirname-->"+str(mainDirname)
print "Filename-->"+str(Filename)
mylog.write("loginfor-->"+str(logInfo)+'\n')
mylog.write("dirInfor-->"+str(dirInfo)+'\n')
mylog.write("Dirname-->"+str(mainDirname)+'\n')
mylog.write("Filename-->"+str(Filename)+'\n')



else:
print "Already downloaded in our local drive" +str(mainDirname)
mylog.write( "Already downloaded in our local drive" +str(mainDirname)+'\n')



finally:
ftp.quit()
ftp.close()
print "Closing FTP"
mylog.write("Closing FTP")
mylog.close()

Simple GUI Demo in Python

Be The First To Comment
from Tkinter import*
mainForm=Tk()

lblWelcome = Label(mainForm, text="Welcome to SSEBB Model")
lblWelcome.pack()

lblDisplay=Listbox(mainForm)

txtMyval=Entry(mainForm)
txtMyval.focus_set()
txtMyval.pack()

btnQuit=Button(mainForm,text="Close",command=quit)
btnQuit.pack(side="left",padx=10,pady=10)

def ShowVal():
lblDisplay.insert(0,txtMyval.get())


btnStart=Button(mainForm,text="Show in list",command=ShowVal)
btnStart.pack()

lblDisplay.pack(padx=20,pady=10)

mainForm.mainloop()
 

© 2011 GIS and Remote Sensing Tools, Tips and more .. ToS | Privacy Policy | Sitemap

About Me