Empowering Knowledge

Script to import HTML files into Zendesk Forums

Weeks ago, I was given with a task to import bunch of HTML files into Zendesk Forum as articles. As there were thousands of files and it’s tough to copy paste them manually. I got to know that Zendesk offers a rich set of API’s to manage Forum and it’s contents. I have created a Python script to import bunch of offline HTML files into Zendesk Forum creating Topics/articles in a Forum under Categories

We can perform either of the following features in Forum section.

List Forums
Show Forum
Create Forum
Update Forum
Delete Forum

Example request available in the Zendesk developers page is using curl.

curl https://yoursubdomain.zendesk.com/api/v2/topics.json -H "Content-Type: application/json" -v -u email:password -X POST -d '{"topic": {"forum_id": 79161, "title": "My Topic", "body":"This is some code", "topic_type": "articles", "access": "logged in users" }'

The above example had a limitation of adding HTML code into<code>, which is impossible when we had thousands of files to be uploaded. Thus I wrote a Python script tweaking the above example, so that I can refer an external file to fetch HTML contents and import it as an individual Topic.

Source can be found here..

https://github.com/ijeyanthan/zendesk-import-into-forum

To know more, visit http://developer.zendesk.com/documentation/rest_api/forums.html for the list of possible JSON requests available in Zendesk.

Java program using Apache Tika to extract text from various formats into a String object

Apache Tika can get the plain text from so many formats like Microsoft's Office files and PDF etc.

The tika app jar file can output plain text from those files and print them into a file or console.

This is a sample java program that uses this tika jar file as library and uses the Parser Api to get the text into a String object.

TextExtractor.java

import java.io.ByteArrayOutputStream;

import java.io.File;

import java.io.InputStream;

import java.io.OutputStream;

import java.io.OutputStreamWriter;

import java.net.URL;

import org.apache.tika.detect.DefaultDetector;

import org.apache.tika.detect.Detector;

import org.apache.tika.io.TikaInputStream;

import org.apache.tika.metadata.Metadata;

import org.apache.tika.parser.AutoDetectParser;

import org.apache.tika.parser.ParseContext;

import org.apache.tika.parser.Parser;

import org.apache.tika.sax.BodyContentHandler;

import org.xml.sax.ContentHandler;

class TextExtractor {

private OutputStream outputstream;

private ParseContext context;

private Detector detector;

private Parser parser;

private Metadata metadata;

private String extractedText;

public TextExtractor() {

context = new ParseContext();

detector = new DefaultDetector();

parser = new AutoDetectParser(detector);

context.set(Parser.class, parser);

outputstream = new ByteArrayOutputStream();

metadata = new Metadata();

}

public void process(String filename) throws Exception {

URL url;

File file = new File(filename);

if (file.isFile()) {

url = file.toURI().toURL();

} else {

url = new URL(filename);

}

InputStream input = TikaInputStream.get(url, metadata);

ContentHandler handler = new BodyContentHandler(outputstream);

parser.parse(input, handler, metadata, context);

input.close();

}

public void getString() {

//Get the text into a String object

extractedText = outputstream.toString();

//Do whatever you want with this String object.

System.out.println(extractedText);

}

public static void main(String args[]) throws Exception {

if (args.length == 1) {

TextExtractor textExtractor = new TextExtractor();

textExtractor.process(args[0]);

textExtractor.getString();

} else {

throw new Exception();

}

}

}

Compile:

javac -cp ".:tika-app-1.0.jar" TextExtractor.java

Run:

java -cp ".:tika-app-1.0.jar" TextExtractor SomeWordDocument.doc

Note: Replace ":" with ";" if you are in Windows

Posted via email from Art, Science & Technology

Python script to create files/folders from a template

I was following this jenkins plugin tutorial today http://javaadventure.blogspot.in/2008/01/writing-hudson-plug-in-part-1.html which required me to create several directory structures. I found it vexing to do them manually. So I got this idea to make a python program that creates directory structures by reading a template file.

We will have to pass it a template file that specifies the directory structure which will be created. The template file should look something like this:

foldername/
    filename.ext
    subfoldername/
        subsubfoldername/
            anotherfile.txt
    someotherfile.xml

It can be anything like that. There are only two syntax requirements.
1.Indentation must be uniform (like in a python program)
2.folder names must end with a '/' otherwise the program can't know which is folder and which is file.

Screenshot:

Code:

#!/usr/bin/python

import sys,os

#I'll be using this as a stack pushing and popping indent levels

indentlevels = [0]

#Opening the file specified in argument

f=open(sys.argv[1])

#If the argument is /hello/structure.txt PARENTPATH should be /hello/

PARENTPATH=os.path.abspath(f.name)

PARENTPATH=PARENTPATH[:PARENTPATH.rindex('/')+1]

#We'll be reading line by line. This variable is used to store previous line.

previous = ""

#Count no. of leading spaces in a line.

#Empty lines give a length of 1 considering '\n' so will return -1 to ignore later

def countSpaces(data):

return len(data)-len(data.lstrip()) if (len(data)>1) else -1

#This function creates the files or folders

def touch(data):

global PARENTPATH

if (data.endswith('/')):

os.mkdir(PARENTPATH+data)

else:

f=open(PARENTPATH+data,"w")

f.close()

#Main program starts.. Iterate through lines

line=f.readline()

while line:

#Get the indent level

indentlevel = countSpaces(line)

#Remove leading and trailing spaces and get only the string

line=line.strip()

#Ignore empty lines and continue the loop

if (indentlevel == -1):

line = f.readline()

continue

#If indent is increased,

if(indentlevel > indentlevels[-1]):

#Check whether the previous line ended with '/'.

#Because we can put files and folders within a folder. Not possible within a file

if not previous.endswith('/'):

print "SYNTAX ERROR.. You can indent a line further only if the above line specifies a folder (ends with /)"

sys.exit()

#add the new indentlevel to indentlevels

indentlevels.append(indentlevel)

#and append the above string as a folder level to the PARENTPATH

PARENTPATH=PARENTPATH+previous

#If indent level is reduced,

elif(indentlevel < indentlevels[-1]):

#pop the last element from the indentlevels

indentlevels.pop() #ref1

#If indent is reduced more than one step (eg. dirstruct4),

#then pop the indentlevels until it's equal to indentlevel

#and do the same for parent path

while indentlevels[-1]>indentlevel:

indentlevels.pop()

PARENTPATH=PARENTPATH[:PARENTPATH[:-1].rindex('/')+1]

# Because I want /hai/bai/ to become /hai/

#If this condition fails, then it means the file is not indented uniformly

if not (indentlevel == indentlevels[-1]):

print "SYNTAX ERROR.. INDENTATION MUST BE UNIFORM"

sys.exit()

#required for line '#ref1'

PARENTPATH=PARENTPATH[:PARENTPATH[:-1].rindex('/')+1]

#else: #indentlevel == indentlevels[-1]

touch(line)

previous=line

line=f.readline()

Try this sample dir structure:

project/

SimpleM2Project/

pom.xml

src/

main/

java/

com/

onedash/

hello/

Hello.java

test/

java/

Posted via email from Art, Science & Technology